Human-Centric Interfaces for Ambient Intelligence

Book description

To create truly effective human-centric ambient intelligence systems both engineering and computing methods are needed. This is the first book to bridge data processing and intelligent reasoning methods for the creation of human-centered ambient intelligence systems. Interdisciplinary in nature, the book covers topics such as multi-modal interfaces, human-computer interaction, smart environments and pervasive computing, addressing principles, paradigms, methods and applications. This book will be an ideal reference for university researchers, R&D engineers, computer engineers, and graduate students working in signal, speech and video processing, multi-modal interfaces, human-computer interaction and applications of ambient intelligence.

Table of contents

  1. Front Cover
  2. Human-Centric Interfaces for Ambient Intelligence
  3. Copyright Page
  4. Contents (1/3)
  5. Contents (2/3)
  6. Contents (3/3)
  7. Foreword
  8. Preface
    1. Ambient Intelligence
    2. Human-Centric Design
    3. Vision and Visual Interfaces
    4. Speech Processing and Dialogue Management
    5. Multimodal Interfaces
    6. Smart Environment Applications
    7. Conclusions
    8. Acknowledgments
  9. Part 1: Vision and Visual Interfaces
    1. Chapter 1: Face-to-Face Collaborative Interfaces
      1. 1.1 Introduction
      2. 1.2 Background
      3. 1.3 Surface User Interface
      4. 1.4 Multitouch (1/2)
      5. 1.4 Multitouch (2/2)
        1. 1.4.1 Camera-Based Systems
        2. 1.4.2 Capacitance-Based Systems
      6. 1.5 Gestural Interaction
      7. 1.6 Gestural Infrastructures
        1. 1.6.1 Gestural Software Support
      8. 1.7 Touch versus Mouse
      9. 1.8 Design Guidelines for SUIs for Collaboration
        1. 1.8.1 Designing the Collaborative Environment
      10. 1.9 Conclusions
      11. References
    2. Chapter 2: Computer Vision Interfaces for Interactive Art
      1. 2.1 Introduction
        1. 2.1.1 A Brief History of (Vision in) Art
      2. 2.2 A Taxonomy of Vision-Based Art
      3. 2.3 Paradigms for Vision-Based Interactive Art (1/2)
      4. 2.3 Paradigms for Vision-Based Interactive Art (2/2)
        1. 2.3.1 Mirror Interfaces
        2. 2.3.2 Performance
      5. 2.4 Software Tools
        1. 2.4.1 Max/MSP, Jitter, and Puredata
        2. 2.4.2 EyesWeb
        3. 2.4.3 processing
        4. 2.4.4 OpenCV
      6. 2.5 Frontiers of Computer Vision
      7. 2.6 Sources of Information
      8. 2.7 Summary
      9. Acknowledgments
      10. References
    3. Chapter 3: Ubiquitous Gaze: Using Gaze at the Interface
      1. 3.1 Introduction
      2. 3.2 The Role of Gaze in Interaction
      3. 3.3 Gaze as an Input Device (1/3)
      4. 3.3 Gaze as an Input Device (2/3)
      5. 3.3 Gaze as an Input Device (3/3)
        1. 3.3.1 Eyes on the Desktop
        2. 3.3.2 Conversation-Style Interaction
        3. 3.3.3 Beyond the Desktop
          1. Ambient Displays
          2. Human–Human Interaction in Ambient Environments
            1. Activity detection
            2. Interest level
            3. Hot spot detection
            4. Participation status
            5. Dialogue acts
            6. Interaction structure
            7. Dominance and influence
      6. 3.4 Mediated Communication
      7. 3.5 Conclusion
      8. References
    4. Chapter 4: Exploiting Natural Language Generation in Scene Interpretation
      1. 4.1 Introduction
      2. 4.2 Related Work
      3. 4.3 Ontology-Based User Interfaces
      4. 4.4 Vision and Conceptual Levels
      5. 4.5 The NLG Module
        1. 4.5.1 Representation of the Discourse
        2. 4.5.2 Lexicalization
        3. 4.5.3 Surface Realization
      6. 4.6 Experimental Results
      7. 4.7 Evaluation
        1. 4.7.1 Qualitative Results
        2. 4.7.2 Quantitative Results
      8. 4.8 Conclusions
      9. Acknowledgments
      10. Appendix Listing of Detected Facts Sorted by Frequency of Use
      11. References
    5. Chapter 5: The Language of Action: A New Tool for Human-Centric Interfaces
      1. 5.1 Introduction
      2. 5.2 Human Action
      3. 5.3 Learning the Languages of Human Action
        1. 5.3.1 Related Work
      4. 5.4 Grammars of Visual Human Movement
      5. 5.5 Grammars of Motoric Human Movement (1/4)
      6. 5.5 Grammars of Motoric Human Movement (2/4)
      7. 5.5 Grammars of Motoric Human Movement (3/4)
      8. 5.5 Grammars of Motoric Human Movement (4/4)
        1. 5.5.1 Human Activity Language: A Symbolic Approach
        2. 5.5.2 A Spectral Approach: Synergies
      9. 5.6 Applications to Health
      10. 5.7 Applications to Artificial Intelligence and Cognitive Systems
      11. 5.8 Conclusions
      12. Acknowledgments
      13. References
  10. Part 2: Speech Processing and Dialogue Management
    1. Chapter 6: Robust Speech Recognition Under Noisy Ambient Conditions
      1. 6.1 Introduction
      2. 6.2 Speech Recognition Overview
      3. 6.3 Variability in the Speech Signal
      4. 6.4 Robust Speech Recognition Techniques (1/3)
      5. 6.4 Robust Speech Recognition Techniques (2/3)
      6. 6.4 Robust Speech Recognition Techniques (3/3)
        1. 6.4.1 Speech Enhancement Techniques
        2. 6.4.2 Robust Feature Selection and Extraction Methods
        3. 6.4.3 Feature Normalization Techniques
        4. 6.4.4 Stereo Data-Based Feature Enhancement
        5. 6.4.5 The Stochastic Matching Framework
          1. Model-Based Model Adaptation
          2. Model-Based Feature Enhancement
          3. Adaptation-Based Compensation
          4. Uncertainty in Feature Enhancement
        6. 6.4.6 Special Transducer Arrangement to Solve the Cocktail Party Problem
      7. 6.5 Summary
      8. References (1/2)
      9. References (2/2)
    2. Chapter 7: Speaker Recognition in Smart Environments
      1. 7.1 Principles and Applications of Speaker Recognition
        1. 7.1.1 Features Used for Speaker Recognition
        2. 7.1.2 Speaker Identification and Verification
        3. 7.1.3 Text-Dependent, Text-Independent, and Text-Prompted Methods
      2. 7.2 Text-Dependent Speaker Recognition Methods
        1. 7.2.1 DTW-Based Methods
        2. 7.2.2 HMM-Based Methods
      3. 7.3 Text-Independent Speaker Recognition Methods
        1. 7.3.1 Methods Based on Long-Term Statistics
        2. 7.3.2 VQ-Based Methods
        3. 7.3.3 Methods Based on Ergodic HMM
        4. 7.3.4 Methods Based on Speech Recognition
      4. 7.4 Text-Prompted Speaker Recognition
      5. 7.5 High-Level Speaker Recognition
      6. 7.6 Normalization and Adaptation Techniques
        1. 7.6.1 Parameter Domain Normalization
        2. 7.6.2 Likelihood Normalization
        3. 7.6.3 HMM Adaptation for Noisy Conditions
        4. 7.6.4 Updating Models and A Priori Thresholds for Speaker Verification
      7. 7.7 ROC and DET Curves
        1. 7.7.1 ROC Curves
        2. 7.7.2 DET Curves
      8. 7.8 Speaker Diarization
      9. 7.9 Multimodal Speaker Recognition
        1. 7.9.1 Combining Spectral Envelope and Fundamental Frequency Features
        2. 7.9.2 Combining Audio and Visual Features
      10. 7.10 Outstanding Issues
      11. References
    3. Chapter 8: Machine Learning Approaches to Spoken Language Understanding for Ambient Intelligence
      1. 8.1 Introduction
      2. 8.2 Statistical Spoken Language Understanding
        1. 8.2.1 Spoken Language Understanding for Slot-Filling Dialogue System
        2. 8.2.2 Sequential Supervised Learning
      3. 8.3 Conditional Random Fields
        1. 8.3.1 Linear-Chain CRFs
        2. 8.3.2 Parameter Estimation
        3. 8.3.3 Inference
      4. 8.4 Efficient Algorithms for Inference and Learning
        1. 8.4.1 Fast Inference for Saving Computation Time
        2. 8.4.2 Feature Selection for Saving Computation Memory
      5. 8.5 Transfer Learning for Spoken Language Understanding
        1. 8.5.1 Transfer Learning
        2. 8.5.2 Triangular-Chain Conditional Random Fields
          1. Model1
          2. Model2
        3. 8.5.3 Parameter Estimation and Inference
      6. 8.6 Joint Prediction of Dialogue Acts and Named Entities
        1. 8.6.1 Data Sets and Experiment Setup
        2. 8.6.2 Comparison Results for Text and Spoken Inputs
        3. 8.6.3 Comparison of Space and Time Complexity
      7. 8.7 Multi-Domain Spoken Language Understanding (1/2)
      8. 8.7 Multi-Domain Spoken Language Understanding (2/2)
        1. 8.7.1 Domain Adaptation
        2. 8.7.2 Data and Setup
        3. 8.7.3 Comparison Results
      9. 8.8 Conclusion and Future Direction
      10. Acknowledgments
      11. References
    4. Chapter 9: The Role of Spoken Dialogue in User-Environment Interaction
      1. 9.1 Introduction
      2. 9.2 Types of Interactive Speech Systems
      3. 9.3 The Components of an Interactive Speech System (1/2)
      4. 9.3 The Components of an Interactive Speech System (2/2)
        1. 9.3.1 Input Interpretation
        2. 9.3.2 Output Generation
        3. 9.3.3 Dialogue Management
      5. 9.4 Examples of Spoken Dialogue Systems for Ambient Intelligence Environments (1/2)
      6. 9.4 Examples of Spoken Dialogue Systems for Ambient Intelligence Environments (2/2)
        1. 9.4.1 Chat
        2. 9.4.2 SmartKom and SmartWeb
        3. 9.4.3 Talk
        4. 9.4.4 Companions
      7. 9.5 Challenges for Spoken Dialogue Technology in Ambient Intelligence Environments
        1. 9.5.1 Infrastructural Challenges
        2. 9.5.2 Challenges for Spoken Dialogue Technology
      8. 9.6 Conclusions
      9. References
    5. Chapter 10: Speech Synthesis Systems in Ambient Intelligence Environments
      1. 10.1 Introduction
      2. 10.2 Speech Synthesis Interfaces for Ambient Intelligence
      3. 10.3 Speech Synthesis (1/2)
      4. 10.3 Speech Synthesis (2/2)
        1. 10.3.1 Text Processing
        2. 10.3.2 Speech Signal Synthesis
          1. Articulatory Synthesis
          2. Formant Synthesis
          3. Concatenative Synthesis
        3. 10.3.3 Prosody Generation
        4. 10.3.4 Evaluation of Synthetic Speech
      5. 10.4 Emotional Speech Synthesis
      6. 10.5 Discussion
        1. 10.5.1 Ambient Intelligence and Users
        2. 10.5.2 Future Directions and Challenges
      7. 10.6 Conclusions
      8. Acknowledgments
      9. References
  11. Part 3: Multimodal Interfaces
    1. Chapter 11: Tangible Interfaces for Ambient Augmented Reality Applications
      1. 11.1 Introduction
        1. 11.1.1 Rationale for Ambient AR Interfaces
        2. 11.1.2 Augmented Reality
      2. 11.2 Related Work
        1. 11.2.1 From Tangibility...
        2. 11.2.2 . . .To the AR Tangible User Interface
      3. 11.3 Design Approach for Tangible AR Interfaces
        1. 11.3.1 The Tangible AR Interface Concept
      4. 11.4 Design Guidelines
      5. 11.5 Case Studies (1/2)
      6. 11.5 Case Studies (2/2)
        1. 11.5.1 AR Lens
        2. 11.5.2 AR Tennis
        3. 11.5.3 MagicBook
      7. 11.6 Tools for Ambient AR Interfaces
        1. 11.6.1 Software Authoring Tools
        2. 11.6.2 Hardware Authoring Tools
      8. 11.7 Conclusions
      9. References
    2. Chapter 12: Physical Browsing and Selection-Easy Interaction with Ambient Services
      1. 12.1 Introduction to Physical Browsing
      2. 12.2 Why Ambient Services Need Physical Browsing Solutions
      3. 12.3 Physical Selection
        1. 12.3.1 Concepts and Vocabulary
        2. 12.3.2 Tou
        3. 12.3.3 Pointing
        4. 12.3.4 Scanning
        5. 12.3.5 Visualizing Physical Hyperlinks
      4. 12.4 Selection as an Interaction Task (1/2)
      5. 12.4 Selection as an Interaction Task (2/2)
        1. 12.4.1 Selection in Desktop Computer Systems
        2. 12.4.2 About the Choice of Selection Technique
        3. 12.4.3 Selection in Immersive Virtual Environments
        4. 12.4.4 Selection with Laser Pointers
        5. 12.4.5 The Mobile Terminal as an Input Device
      6. 12.5 Implementing Physical Selection (1/2)
      7. 12.5 Implementing Physical Selection (2/2)
        1. 12.5.1 Implementing Pointing
        2. 12.5.2 Implementing Touching
          1. RFID as an Implementation Technology
          2. User Interaction Considerations
        3. 12.5.3 Other Technologies for Connecting Physical and Digital Entities
          1. Visual Technologies for Mobile Terminals
          2. Body Communication
      8. 12.6 Indicating and Negotiating Actions After the Selection Event
        1. 12.6.1 Activation by Selection
        2. 12.6.2 Action Selection by a Different Modality
        3. 12.6.3 Actions by Combining Selection Events
        4. 12.6.4 Physical Selection in Establishing Communication
      9. 12.7 Conclusions
      10. References
    3. Chapter 13: Nonsymbolic Gestural Interaction for Ambient Intelligence
      1. 13.1 Introduction
      2. 13.2 Classifying Gestural Behavior for Human-Centric Ambient Intelligence
      3. 13.3 Emotions
      4. 13.4 Personality
      5. 13.5 Culture
      6. 13.6 Recognizing Gestural Behavior for Human-Centric Ambient Intelligence
        1. 13.6.1 Acceleration-Based Gesture Recognition
        2. 13.6.2 Gesture Recognition Based on Physiological Input
      7. 13.7 Conclusions
      8. References
    4. Chapter 14: Evaluation of Multimodal Interfaces for Ambient Intelligence
      1. 14.1 Introduction
      2. 14.2 Performance and Quality Taxonomy
      3. 14.3 Quality Factors
      4. 14.4 Interaction Performance Aspects
      5. 14.5 Quality Aspects
      6. 14.6 Application Examples (1/3)
      7. 14.6 Application Examples (2/3)
      8. 14.6 Application Examples (3/3)
        1. 14.6.1 INSPIRE and MediaScout
        2. 14.6.2 Evaluation Constructs
        3. 14.6.3 Evaluation of Output Metaphors
          1. Rationale
          2. Experimental Design
          3. Insights
        4. 14.6.4 Evaluation of the Quality of an Embodied Conversational Agent
          1. Rationale
          2. Experimental Design
          3. Insights
        5. 14.6.5 Comparison of Questionnaires
          1. Rationale
          2. Experimental Design
          3. Insights
            1. Comparison of questionnaire results
            2. Comparison of quality and performance metrics
      9. 14.7 Conclusions and Future Work
      10. Acknowledgment
      11. References
  12. Part 4: Smart Environment Applications
    1. Chapter 15: New Frontiers in Machine Learning for Predictive User Modeling
      1. 15.1 Introduction
        1. 15.1.1 Multimodal Affect Recognition
        2. 15.1.2 Modeling Interruptability
        3. 15.1.3 Classifying Voice Mails
        4. 15.1.4 Brain–Computer Interfaces for Visual Recognition
      2. 15.2 A Quick Primer: Gaussian Process Classification
      3. 15.3 Sensor Fusion
        1. 15.3.1 Multimodal Sensor Fusion for Affect Recognition
        2. 15.3.2 Combining Brain–Computer Interface with Computer Vision
      4. 15.4 Semisupervised Learning
        1. 15.4.1 Semisupervised Affect Recognition
      5. 15.5 Active Learning
        1. 15.5.1 Modeling Interruptability
        2. 15.5.2 Classifying Voice Mails
      6. 15.6 Conclusions
      7. Acknowledgments
      8. References
    2. Chapter 16: Games and Entertainment in Ambient Intelligence Environments
      1. 16.1 Introduction
      2. 16.2 Ambient Entertainment Applications
        1. 16.2.1 Ubiquitous Devices
        2. 16.2.2 Exergames
        3. 16.2.3 Urban Gaming
        4. 16.2.4 Dancing in the Streets
      3. 16.3 Dimensions in Ambient Entertainment (1/2)
      4. 16.3 Dimensions in Ambient Entertainment (2/2)
        1. 16.3.1 Sensors and Control
        2. 16.3.2 Location
        3. 16.3.3 Social Aspects of Gaming
      5. 16.4 Designing for Ambient Entertainment and Experience (1/2)
      6. 16.4 Designing for Ambient Entertainment and Experience (2/2)
        1. 16.4.1 Emergent Games
        2. 16.4.2 Rhythm and Temporal Interaction
        3. 16.4.3 Performance in Play
        4. 16.4.4 Immersion and Flow
      7. 16.5 Conclusions
      8. Acknowledgments
      9. References
    3. Chapter 17: Natural and Implicit Information-Seeking Cues in Responsive Technology
      1. 17.1 Introduction
      2. 17.2 Information Seeking and Indicative Cues
        1. 17.2.1 Analysis of the Hypothetical Shopping Scenario
        2. 17.2.2 A Framework for Information Seeking
        3. 17.2.3 Indicative Cues by Phase
      3. 17.3 Designing Systems for Natural and Implicit Interaction
        1. 17.3.1 Natural Interaction
        2. 17.3.2 Implicit Interaction
      4. 17.4 Clothes Shopping Support Technologies
        1. 17.4.1 Fitting Room Technologies
        2. 17.4.2 Virtual Fittings
        3. 17.4.3 Reactive Displays
      5. 17.5 Case Study: Responsive Mirror (1/5)
      6. 17.5 Case Study: Responsive Mirror (2/5)
      7. 17.5 Case Study: Responsive Mirror (3/5)
      8. 17.5 Case Study: Responsive Mirror (4/5)
      9. 17.5 Case Study: Responsive Mirror (5/5)
        1. 17.5.1 Concept
        2. 17.5.2 Privacy Concerns
          1. Disclosure
          2. Identity
          3. Temporal
        3. 17.5.3 Social Factors: Reflecting Images of Self and Others
        4. 17.5.4 Responsive Mirror Prototype
        5. 17.5.5 Vision System Description
          1. Shopper Detection
          2. Orientation Estimation
          3. Clothes Recognition
            1. Subjectivity of Clothing Similarity
            2. Clothing Similarity Algorithm
            3. Feature Extraction—Shirt Parts Segmentation
            4. Feature Extraction—Sleeve Length Detection
            5. Feature Extraction—Collar Detection
            6. Feature Extraction—Button Detection
            7. Feature Extraction—Pattern Detection
            8. Feature Extraction—Emblem Detection
        6. 17.5.6 Design Evaluation
          1. Method
          2. Task and Procedure
          3. Results
          4. Fitting Room Behavior
          5. User Suggestions for Enhancement
          6. Use of Images of Other People
          7. Results from Privacy-Related Questions
      10. 17.6 Lessons for Ambient Intelligence Designs of Natural and Implicit Interaction
      11. Acknowledgments
      12. References
    4. Chapter 18: Spoken Dialogue Systems for Intelligent Environments
      1. 18.1 Introduction
      2. 18.2 Intelligent Environments (1/2)
      3. 18.2 Intelligent Environments (2/2)
        1. 18.2.1 System Architecture
        2. 18.2.2 The Role of Spoken Dialogue
          1. Network Speech Recognition
          2. Distributed Speech Recognition
            1. ETSI DSR front-end standards
            2. A Java ME implementation of the DSR front-end
        3. 18.2.3 Proactiveness
      4. 18.3 Information Access in Intelligent Environments (1/3)
      5. 18.3 Information Access in Intelligent Environments (2/3)
      6. 18.3 Information Access in Intelligent Environments (3/3)
        1. 18.3.1 Pedestrian Navigation System
          1. System Description
          2. Evaluation
        2. 18.3.2 Journey-Planning System
        3. 18.3.3 Independent Dialogue Partner
          1. Proactive Dialogue Modeling
          2. Usability Evaluation
      7. 18.4 Conclusions
      8. Acknowledgments
      9. References
    5. Chapter 19: Deploying Context-Aware Health Technology at Home: Human-Centric Challenges
      1. 19.1 Introduction
      2. 19.2 The Opportunity: Context-Aware Home Health Applications
        1. 19.2.1 Medical Monitoring
        2. 19.2.2 Compensation
        3. 19.2.3 Prevention
        4. 19.2.4 Embedded Assessment
      3. 19.3 Case Study: Context-Aware Medication Adherence
        1. 19.3.1 Prototype System
        2. 19.3.2 Evaluation
        3. 19.3.3 Human-Centric Design Oversights
      4. 19.4 Detecting Context: Twelve Questions to Guide Research (1/3)
      5. 19.4 Detecting Context: Twelve Questions to Guide Research (2/3)
      6. 19.4 Detecting Context: Twelve Questions to Guide Research (3/3)
        1. 19.4.1 Sensor Installation (“Install It”)
          1. Question 1: What Type of Sensors Will Be Used?
          2. Question 2: Are the Sensors Professionally Installed or Self-Installedin the Home?
          3. Question 3: What Is the Cost of (end-user) Installation?
          4. Question 4: Where Do Sensors Need to Go?
          5. Question 5: How Are Sensors Selected, Positioned, and Labeled?
            1. Selection
            2. Positioning
            3. Labeling
        2. 19.4.2 Activity Model Training (“Customize It”)
          1. Question 6: What Type of Training Data Do the Activity Models Require?
          2. Question 7: How Many Examples Are Needed?
        3. 19.4.3 Activity Model Maintenance (“Fix It”)
          1. Question 8: Who Will Maintain the System as Activities Change, the Environment Changes, and Sensors Break?
          2. Question 9: How Does the User Know What Is Broken?
          3. Question 10: Can the User Make Instantaneous, Nonoscillating Fixes?
          4. Question 11: What Will Keep the User’s Mental Model in Line with the Algorithmic Model?
          5. Question 12: How Does a User Add a New Activity to Recognize?
      7. 19.5 Conclusions
      8. Acknowledgments
      9. References
  13. Epilogue: Challenges and Outlook (1/2)
  14. Epilogue: Challenges and Outlook (2/2)
  15. Index

Product information

  • Title: Human-Centric Interfaces for Ambient Intelligence
  • Author(s): Hamid Aghajan, Juan Carlos Augusto, Ramon Lopez-Cozar Delgado
  • Release date: October 2009
  • Publisher(s): Academic Press
  • ISBN: 9780080878508