CHAPTER 8
From Annotation to
Multimodal Behavior
Kristiina Jokinen and Catherine Pelachaud
1. Introduction
The usage of multimodal corpora is crucial for studying human
behavior as well as modeling embodied conversational agents (Cowie
et al., 2010a). Gathering data and creating corpora is complex.
Interested readers can read work by Cowie and Douglas-Cowie on
the topic (Cowie et al., 2010b) wherein different methodologies for
creating corpora are described.
Once the corpora are gathered, there are the issues of annotating
them. Defining an annotation schema is very much in relation to the
phenomenon to be studied. When defining, it is necessary to decide
which information should be marked and in what form, whether it
should be continuous values ...