Making Sense of Sensors: End-to-End Algorithms and Infrastructure Design from Wearable-Devices to Data Centers

© Omesh Tickoo and Ravi Iyer 2017

Omesh Tickoo and Ravi Iyer, Making Sense of Sensors, 10.1007/978-1-4302-6593-1_3

3. Multimodal Recognition

Omesh Tickoo¹  and Ravi Iyer²

(1)Portland, Oregon, USA

(2)3585 SW, Portland, USA

In the last chapter, we introduced processing of each type of raw sensor data independently to realize different levels of recognition capabilities. We found that inertial sensors provide information about movement across different axes. Audio streams can be processed not only for sound types but also words and natural language. We also showed how to recognize different objects, faces, activities, and more. Now we imagine what is possible if we were able to correlate the understanding across multiple sensor types simultaneously. This ...