Designing Across Senses

Chapter 2. The Structure of Multimodal Experiences

“Our personal worlds are constructions built by our brains using the raw materials of the senses—raw materials that are greatly modified during the construction process.”

—FAITH HICKMAN BRYNIE IN BRAIN SENSE (2009)

WHILE THE TITLE USER experience designer implies that designers create experiences for people, it might be more accurate to say that people construct experiences for themselves. As the saying goes, “Beauty is in the eye of the beholder.” And not just beauty, but functionality, clarity, and reliability. If you want to be very literal, they aren’t just in our eye. They are also in our skin, distributed across the surface of our bones, within our spines, and definitely a bunch of it is up in our heads.

Designers, magicians, and filmmakers tap into many aspects of this construction process to great effect. Our eyes can differentiate hundreds of frames per second, but we can extrapolate continuous movements at much lower speeds, which allows us to enjoy stop-motion animation and animated gifs. Many graphical user interfaces, or GUIs, use the metaphor of the desktop as a way to organize information (see Figure 2-1). This relies on a process known as assimilation, our ability to reuse mental models across new types of information. We have already learned how a physical desk works; this existing mental model helps us to understand how to use a computer’s operating system more easily. Another perceptual phenomenon, ...