In the first television advertisement for the iPad, the narrator intoned, “It’s crazy powerful. It’s magical. You already know how to use it.� This was an astonishing claim. Here was a new, market-defining, revolutionary device, unlike anything we had seen before, and we already knew how to use it. And yet, for the most part, the claim was true. How does a company like Apple make such great new things that people already know how to use?

One answer lies in the ability of Apple designers to draw upon patterns that people are familiar with. The interaction medium might be completely new: before the iPhone, few people had used a multitouch screen. But everyone knew how to pinch or stretch something, and this interaction pattern was easily transferrable to the small screen after seeing it done just once. As Alan Cooper writes in About Face, “All idioms must be learned; good idioms need to be learned only once.�

The Role Of Dopamine In Pattern Recognition

Our brains like to find such patterns. We are wired to search for patterns that our past experiences have shown will lead to successful interactions (in love, war, gambling, investing, etc.). Jonah Lehrer, in How We Decide, writes that our brain produces a pleasure-inducing neurochemical, dopamine, when we recognize familiar patterns in the world around us. When we act on these patterns and are successful in whatever we are trying to do, we get an additional burst of this pleasing chemical.

If we think we recognize a pattern but are mistaken, or if the pattern doesn’t behave in the way we expect it to, then we do not get that second infusion of the neurochemical, and we readjust our expectations. Many neuroscientists believe this reward system is one way in which learning takes place. The process creates a self-reinforcing, pleasure-based cycle that encourages us to learn from our mistakes and to become better interpreters of the world around us.

The dopamine reward system produces positive or negative emotions based on our experiences in the world. Lehrer argues that this reverses our age-old understanding of the role of emotions in the decision-making process. Since Plato, the rational mind has been depicted as the charioteer holding the reins on our unruly emotions. What makes humans unique, according to this metaphor, is our ability to use logic and rationality to control our emotions and make rational decisions. Lehrer’s book details recent research in neuroscience that upends this reason-based model of decision-making. Emotions, some of them caused by the dopamine-based reward system, play a central role in our decision-making processes.

Brain
(Image: Sue Clark)

These discoveries in neuroscience provide a strong argument for using design patterns in interaction design. Take the carousel pattern, which is prevalent in desktop, tablet and handheld devices. The Yahoo Design Library has a useful illustration of this design pattern. Content appears to slide in from one side of the panel; items at each end are partially obscured to indicate that more virtual space, and more content, lies outside the carousel pane; arrows appear, when appropriate, to indicate how to get to that additional content. This is a very simple pattern that people can learn after using the feature just once.

New users of Pandora will encounter this carousel pattern almost to the letter, and even if they are encountering it for the first time, they will learn it almost immediately. Then, when they encounter versions of the carousel pattern in other designs, they will recognize it before they even begin to interact with it. Their recognition of the pattern will produce pleasure as the dopamine neurons begin firing. When the user then interacts with the pattern — by clicking the arrow on either end to reveal additional content, for instance — and is successful, then more dopamine is produced, leading to additional feelings of pleasure.

Carousel
Carousel design pattern, via Yahoo Developer Network

Admittedly, neuroscientists have not yet attached functional magnetic resonance imaging machines to users in order to measure their brain’s dopamine production as they experience the carousel (or any other) interaction design pattern. To date, our insight into the brain’s responses to the patterns we encounter in the world is limited to what we can extrapolate to humans from experiments that have been conducted on monkeys and to inferences we can draw from the work of psychologists.

Lehrer’s radar technician’s story

Lehrer tells the story of a radar technician during the first Gulf War who spent several days watching blips that represented fighter aircraft returning to ship from a certain point on the coast of Kuwait. One set of blips in the early morning made the technician feel nervous, and he couldn’t explain why. They looked to him to be just the same as those he had observed hour after hour in days past, but his emotional response to this particular set of early morning blips told him that something was wrong. Acting on little more than this emotional response, he ordered the blips to be shot down — thus saving countless lives: the blips turned out to be enemy missiles en route to destroy Allied ships in the Gulf.

The technician could not explain how he knew they were not just another pair of fighter jets. It was only after much review and the discoveries of a cognitive psychologist who was brought in to review the case that investigators determined that what was different about those blips was where they first appeared on his screen: a little farther from shore than all of the other blips. He couldn’t tell at the time that this is what made them different, but subconsciously his brain detected a change in the pattern that he had been observing for hours. The change in pattern caused an emotional and somatic response of panic and anxiety and caused him, despite his reason, to order the blips to be destroyed.

The radar technician’s story (and many others recounted in Lehrer’s book) suggests that our brains observe and act on patterns without our being conscious of it. Recognizable patterns appear, our dopamine neurons fire, our learning is reinforced, and we remain in a state of “flow.� But when a pattern is broken or behaves unexpectedly, all hell breaks loose. Our brain sends out a “prediction error signal�. An area of the brain called the anterior cingulate cortex (ACC) monitors the activity of the prefrontal cortex, and when the ACC detects the absence of activity among dopamine neurons that results from the predicted event not occurring, it sends out this error signal. This results in other chemicals being produced, by the amygdala and the hypothalamus, among other areas, which causes these feelings of panic and anxiety: the heart races, the muscles tense, we become short of breath.

Broken Patterns Cause Panic And Anxiety

Ordinarily, we do not want our users to experience these feelings of panic and anxiety when they use our systems. Yet we know it happens frequently. One reason is that we often present users with interfaces that lack the visual cues to indicate what patterns are being employed. Consider Roku’s Channel Store. When users visit the interface to add a channel to their system, they are confronted with what appears to be a static table of contents. Without prior experience of the carousel pattern, users might interpret this 3 × 4 tabular interface as offering only 12 channels.

In fact, this table does behave according to the carousel pattern. Additional content does lie to the right and left of each row. The content even scrolls vertically as well, but users would never know this from the visual display of the information. Even worse, a new user will learn little about the carousel pattern to apply to their next encounter with it. Ironically, Roku is best known as a Netflix streamer, and Netflix itself applies the carousel pattern expertly to its similar table of contents in its streaming interfaces on game devices such as the Wii. Way back in Design of Everyday Things, Donald Norman defined “visibility� as meaning that “the correct controls are visible, and they convey the correct information.� Neither is the case in the design of Roku’s Channel Store, so users have no way of knowing, without extensive exploration, that the carousel pattern is being employed.

Roku Channel Store
Roku’s Channel Store.

Sometimes, the problem is the reverse: users will think a design pattern is being used when it isn’t. What we recognize as a pattern doesn’t function as we expect; our brains think that something has gone wrong, and the result, again, is anxiety and panic. Take the basic design of a list of items on a smartphone. Users of iOS know this pattern well; it is famously illustrated in Josh Clark’s Tapworthy: Designing Great iPhone Apps. A left-to-right swipe gesture opens a control for deletion, prompting the user to confirm the delete action. This design pattern is easy to learn, but its implementation in other smartphone applications is sporadic and unpredictable. Palm’s webOS email system, for example, uses the swipe gesture for deletion but offers no “Delete� button to confirm the gesture. The email item simply vanishes off the screen. In the messaging application on Palm’s OS, on the other hand, the system does present a deletion control.

iOS Swipe
Swiping left to right to delete in iOS. (Image: Josh Clark)

Early versions of the Android OS didn’t acception the swipe gesture for deletion at all, and it usually interpreted the gesture as a tap by opening the “Edit Itemâ€� page. The Gingerbread update introduced even more inconsistency to the user experience: a right-to-left swipe over a contact, for instance, opens the instant messaging app, and a left-to-right swipe opens the phone app — and initiates a phone call! A user who would naturally expect this gesture to trigger a prompt to delete the contact suddenly finds themselves calling that contact. Talk about panic!

Pattern-Matching Is Harder Than It Sounds

All of us have experienced this feeling of panic to one degree or another. I still feel it when I instinctively move my mouse (in Windows) to the task bar to return to a Web page that I thought I had minimized, when in fact (and for at least three years now) the page I am looking for is open in a different tab, rather than in a minimized, separate window. Interaction habits of mind do not change quickly. And because I use three different Web browsers on at least four different computers, I am constantly unsettled in my search for the “Home� button, which used to be to the left of the URL window in most browsers, but now is all the way on the right in the standard installation of Firefox 12 on both Windows and Mac and doesn’t exist at all in a standard installation of Safari. There is no longer a reliable pattern to determine where I will find the “Home� button on a Web browser. But my brain wants one, feels good when it finds one and rebels (chemically) when it doesn’t.

To be sure, inconsistencies across platforms, browsers and software can have many causes, from patent issues to design legacies. And it is inevitable that interaction designs will change and improve over time. We should not be held to existing patterns just because the human brain prefers it. But we can design according to our developing understanding of how the brain functions. We can employ idioms, such as “pinch,� that are not obvious but are quickly learned. We can progress gradually, building on fundamental elements of existing designs so that new interaction designs retain enough of the old that our brains still recognize them. We can also cautiously introduce new schemes as redundant elements: one doesn’t have to use three- and four-finger swipe gestures on the MacBook Pro’s mousepad, but once one discovers these gestures, they are easy to adopt as natural improvements to the pointer controls and buttons in application interfaces.

In fact, this last approach takes advantage of the brain’s chemistry. The prediction error signal is sent when an expected event does not occur and the result is disappointment or failure. But sometimes, the result of a prediction error is delight, not panic. The expected result did not occur, but something better did. David Rock, in Your Brain at Work, observes that this experience of delight or novelty also produces dopamine and feelings of pleasure. The experience is similar to that of humor: jokes often work because the punchline presents an unexpected twist, a novel outcome. More importantly, jokes work best when the stakes are minimal: no one really gets hurt in a pratfall. When jokes cut too close to the bone, they are painful. We cringe.

New interaction designs can be introduced according to the same principle: if they cause unexpected delight, and no one (and no one’s data) gets hurt, they will induce unexpected pleasure and will be quickly adopted over the legacy designs they are meant to replace.

Further Resources

(al)

© Charles Hannon for Smashing Magazine, 2012.