The Uncanny Valley in Games and Animation

71

Chapter 4

Uncanny Facial

Expression of Emotion

Words are not emotions, but representations of emotions.

EKMAN 2004, p. 45

T

    U M  (Tinwell,

Grimshaw and Williams, 2010) that I discussed in the previous chap-

ter indicated that a perceived lack of human- likeness in facial expression

exaggerated the uncanny in virtual characters with a human- like appear-

ance. Protagonist characters that were intended to be perceived as empa-

thetic, such as Alex Shepherd from Alone in the Dark (Atari Interactive,

Inc. 2009) and Mary Smith from e Casting (Quantic Dream, 2006), were

regarded as less familiar and human- like when they were judged to portray

a lack of facial emotional expressivity. is eect was particularly salient

in the upper facial region, including the eyebrows and forehead. While this

nding provided some evidence as to which factors evoked perception of

the uncanny, I wanted to know more. It is well established that in humans

facial expression is used not only to communicate how one is feeling but

also as a way to determine the aective state and possible actions of oth-

ers (Darwin, 1872; Ekman, 1979, 1992). Importantly, each emotion type

serves a dierent adaptive function as part of social or survival interaction.

erefore, we may respond more negatively to perception of anger or fear

in another yet more positively to perceived happiness. Based on this, I pon-

dered whether the uncanny eect would be the same or dierent across all

72 ◾ The Uncanny Valley in Games and Animation

emotions. If response to the uncanny was to dier across dierent emotion

types, then why would this occur? Also, what would the implications of

this be for designers when modeling dierent emotion types in a charac-

ter’s facial expression? To begin toward working to nd an answer to these

questions, I set about a new empirical study to investigate more closely

how inadequate movement in the upper face may inuence perception of

a character’s emotive state and the uncanny across the six basic emotions

(Ekman, 1972). is chapter provides a description of the methodology,

design and ndings from that experiment to investigate the implications

of a lack of nonverbal communication (NVC) in the upper face and viewer

perception of uncanniness in virtual characters. I also take the opportu-

nity to describe the origins and adaptive functions of facial expression in

primates that help to set this experiment in context. NVC takes place while

we speak, and the upper facial region plays an integral part in this process.

An overview is provided of the purpose of NVC in humans and the associ-

ated roles of the eyebrows, eyelids and forehead and how they may work

independently or alongside speech and other body and facial movements.

As a preeminent psychologist in the study of emotion types and their

relation to human facial expression, in 2009 Time magazine listed Dr.Paul

Ekman as one of the 100 most inuential people in the world. As a result of

nearly a lifetime’s work, Ekman and his colleague Wallace Friesen devised

a conceptual framework of the muscles involved in creating dierent

expressions at diering intensity. is classication system of the facial

muscles is referred to as the facial action coding system (FACS) and is

commonly used as the blueprint of many modern facial modeling and ani-

mation soware packages. I provide a retrospective of Ekman’s work con-

cerning the facial actions used in NVC and the implications of voluntary

versus involuntary facial movements when detecting “false” expressions.

Consideration is given to the consequences of a perceived lack of NVC in

a human- like character’s facial expression with and without speech. e

main focus of this chapter and the arguments that I make about the pos-

sible cause of the uncanny in human- like virtual characters in this book

are centered on a character’s facial expression, especially the upper facial

region. While this is an important factor in understanding the Uncanny

Valley phenomenon, I acknowledge that facial expression should not be

studied in isolation aside from other bodily movements and speech. e

stimuli used in my experiments have featured vocalization narration,

yet tilts of the head and gesture are integral parts of NVC in social com-

munication. I am hopeful that the discussion of issues regarding facial