Chapter 4
Uncanny Facial
Expression of Emotion
Words are not emotions, but representations of emotions.
EKMAN 2004, p. 45
    U M  (Tinwell,
Grimshaw and Williams, 2010) that I discussed in the previous chap-
ter indicated that a perceived lack of human- likeness in facial expression
exaggerated the uncanny in virtual characters with a human- like appear-
ance. Protagonist characters that were intended to be perceived as empa-
thetic, such as Alex Shepherd from Alone in the Dark (Atari Interactive,
Inc. 2009) and Mary Smith from e Casting (Quantic Dream, 2006), were
regarded as less familiar and human- like when they were judged to portray
a lack of facial emotional expressivity. is eect was particularly salient
in the upper facial region, including the eyebrows and forehead. While this
nding provided some evidence as to which factors evoked perception of
the uncanny, I wanted to know more. It is well established that in humans
facial expression is used not only to communicate how one is feeling but
also as a way to determine the aective state and possible actions of oth-
ers (Darwin, 1872; Ekman, 1979, 1992). Importantly, each emotion type
serves a dierent adaptive function as part of social or survival interaction.
erefore, we may respond more negatively to perception of anger or fear
in another yet more positively to perceived happiness. Based on this, I pon-
dered whether the uncanny eect would be the same or dierent across all
72 The Uncanny Valley in Games and Animation
emotions. If response to the uncanny was to dier across dierent emotion
types, then why would this occur? Also, what would the implications of
this be for designers when modeling dierent emotion types in a charac-
ter’s facial expression? To begin toward working to nd an answer to these
questions, I set about a new empirical study to investigate more closely
how inadequate movement in the upper face may inuence perception of
a character’s emotive state and the uncanny across the six basic emotions
(Ekman, 1972). is chapter provides a description of the methodology,
design and ndings from that experiment to investigate the implications
of a lack of nonverbal communication (NVC) in the upper face and viewer
perception of uncanniness in virtual characters. I also take the opportu-
nity to describe the origins and adaptive functions of facial expression in
primates that help to set this experiment in context. NVC takes place while
we speak, and the upper facial region plays an integral part in this process.
An overview is provided of the purpose of NVC in humans and the associ-
ated roles of the eyebrows, eyelids and forehead and how they may work
independently or alongside speech and other body and facial movements.
As a preeminent psychologist in the study of emotion types and their
relation to human facial expression, in 2009 Time magazine listed Dr.Paul
Ekman as one of the 100 most inuential people in the world. As a result of
nearly a lifetime’s work, Ekman and his colleague Wallace Friesen devised
a conceptual framework of the muscles involved in creating dierent
expressions at diering intensity. is classication system of the facial
muscles is referred to as the facial action coding system (FACS) and is
commonly used as the blueprint of many modern facial modeling and ani-
mation soware packages. I provide a retrospective of Ekmans work con-
cerning the facial actions used in NVC and the implications of voluntary
versus involuntary facial movements when detecting “false” expressions.
Consideration is given to the consequences of a perceived lack of NVC in
a human- like character’s facial expression with and without speech. e
main focus of this chapter and the arguments that I make about the pos-
sible cause of the uncanny in human- like virtual characters in this book
are centered on a character’s facial expression, especially the upper facial
region. While this is an important factor in understanding the Uncanny
Valley phenomenon, I acknowledge that facial expression should not be
studied in isolation aside from other bodily movements and speech. e
stimuli used in my experiments have featured vocalization narration,
yet tilts of the head and gesture are integral parts of NVC in social com-
munication. I am hopeful that the discussion of issues regarding facial

Get The Uncanny Valley in Games and Animation now with O’Reilly online learning.

O’Reilly members experience live online training, plus books, videos, and digital content from 200+ publishers.