CHAPTER 15
Multimodal Fusion in
Human-Agent Dialogue
Elisabeth André, Jean-Claude Martin,
Florian Lingenfelser and Johannes Wagner
1. Introduction
Sophisticated fusion techniques are an essential component of any
multimodal system. Historically, systems aimed at analyzing the
semantics of multimodal commands and typically investigated a
combination of pointing and drawing gestures and speech. The
most prominent example includes the “Put-that-there” system (Bolt,
1980) that analyzes speech in combination with 3D pointing gestures
referring to objects on a graphical display. Since this groundbreaking
work, numerous researchers have investigated mechanisms for
multimodal input interpretation mainly working on speech, gestures
and gaze while the ...