CHAPTER 7
The Situated Multimodal Facets
of Human Communication
Anna Esposito
1. Introduction
Humans interact with each other through a gestalt of emotionally
cognitive actions which involve much more than the speech production
system. In particular, in human interaction, the verbal and nonverbal
communication modes seem to cooperate jointly in assigning semantic
and pragmatic contents to the conveyed message by unraveling
the participants’ cognitive and emotional states and allowing the
exploitation of this information to tailor the interactional process.
These multimodal signals consist of visual and audio information
that singularly or combined may characterize relevant actions for
collaborative learning, shared understanding, decision making ...