Designing for Gesture and Tangible Interaction

Book description

Interactive technology is increasingly integrated with physical objects that do not have a traditional keyboard and mouse style of interaction, and many do not even have a display. These objects require new approaches to interaction design, referred to as post-WIMP (Windows, Icons, Menus, and Pointer) or as embodied interaction design.

This book provides an overview of the design opportunities and issues associated with two embodied interaction modalities that allow us to leave the traditional keyboard behind: tangible and gesture interaction. We explore the issues in designing for this new age of interaction by highlighting the significance and contexts for these modalities. We explore the design of tangible interaction with a reconceptualization of the traditional keyboard as a Tangible Keyboard, and the design of interactive three-dimensional (3D) models as Tangible Models. We explore the design of gesture interaction through the design of gesture-base commands for a walk-up-and-use information display, and through the design of a gesture-based dialogue for the willful marionette. We conclude with design principles for tangible and gesture interaction and a call for research on the cognitive effects of these modalities.

Table of contents

  1. Cover
  2. Copyright
  3. Title Page
  4. Abstract
  5. Contents
  6. Preface
  7. Acknowledgments
  8. 1 Introduction
    1. 1.1 Embodiment
    2. 1.2 Affordance
    3. 1.3 Metaphor
    4. 1.4 Epistemic Actions
    5. 1.5 This Book
  9. 2 Tangible Interaction Design
    1. 2.1 What is a Tangible Interaction?
      1. 2.1.1 Tangible Keyboard
      2. 2.1.2 Tangible Models
    2. 2.2 Why is Tangible Interaction Interesting?
      1. 2.2.1 Space-Multiplexed Input and Output
      2. 2.2.2 Concurrency
      3. 2.2.3 Strong Specific Devices
      4. 2.2.4 Spatially Aware Computational Devices
      5. 2.2.5 Spatial Reconfigurability of Devices
    3. 2.3 What Are Key Design Issues for Tangible User Interfaces?
      1. 2.3.1 Designing Interplay of Virtual and Physical
      2. 2.3.2 Selecting from Multiple Gestures and Actions
      3. 2.3.3 Crossing Disciplinary Boundaries
      4. 2.3.4 The Lack of Standard Input and Output Interaction Models
  10. 3 Designing for Tangible Interaction
    1. 3.1 Designing the Tangible Keyboard
      1. 3.1.1 Context of Use
      2. 3.1.2 Design Goals
      3. 3.1.3 Technology Selection
      4. 3.1.4 Design Methods
      5. 3.1.5 Interaction Model
      6. 3.1.6 Design Heuristics
    2. 3.2 Designing Tangible Models
      1. 3.2.1 Context of Use
      2. 3.2.2 Design Goals
      3. 3.2.3 Technology Selection
      4. 3.2.4 Design Methods
      5. 3.2.5 Interaction Model
      6. 3.2.6 Design Guidelines
  11. 4 Gesture-based Interaction
    1. 4.1 What Is a Gesture-based Interaction?
      1. 4.1.1 Gesture Commands: Walk-Up-and-Use Information Display
      2. 4.1.2 Gesture dialogue: The Willfull Marionette
    2. 4.2 Why is Gesture-based Interaction Interesting?
      1. 4.2.1 Public Walk-Up-and-Use Displays or Objects
      2. 4.2.2 Remote Actuators, Sensors, and Displays
      3. 4.2.3 Sociable Multiple-User Interaction
      4. 4.2.4 Gesture Communication
    3. 4.3 What Are Key Design Issues for Gesture-based Interaction?
      1. 4.3.1 Considering How to Engage People
      2. 4.3.2 Designing Clear and Consistent Visual Feedback
      3. 4.3.3 Selecting Ergonomic Gestures
      4. 4.3.4 Designing Discoverable Gestures
      5. 4.3.5 Inaccurate Gesture Recognition
  12. 5 Designing for Gesture Interaction
    1. 5.1 Designing the Walk-Up-and-Use Information Display
      1. 5.1.1 Context of Use
      2. 5.1.2 Design Goals
      3. 5.1.3 Technology Selection
      4. 5.1.4 Design Methods
      5. 5.1.5 Interaction Model
      6. 5.1.6 Design Heuristics
    2. 5.2 Designing the Willful Marionette
      1. 5.2.1 Context of Use
      2. 5.2.2 Design Goals
      3. 5.2.3 Technology Selection
      4. 5.2.4 Design Methods
      5. 5.2.5 Interaction Model
      6. 5.2.6 Design Guidelines
  13. 6 Looking to the Future: Research Challenges
    1. 6.1 Design Challenges for Embodied Interaction
      1. 6.1.1 Understanding Embodied Actions as User Input
      2. 6.1.2 Designing Gestures with Effective Visual Feedback
      3. 6.1.3 Understanding Sensing Technology
    2. 6.2 Research Challenges for Embodied Interction
      1. 6.2.1 Gesture Design
      2. 6.2.2 Multidisciplinary Research
      3. 6.2.3 Finding Appropriate Contexts of Use
      4. 6.2.4 Limited Scope of Representation
      5. 6.2.5 Cognitive and Physical Fatigue
  14. Bibliography
  15. Authors’ Biographies

Product information

  • Title: Designing for Gesture and Tangible Interaction
  • Author(s): Mary Lou Maher, Lina Lee, John M. Carroll
  • Release date: March 2017
  • Publisher(s): Morgan & Claypool Publishers
  • ISBN: 9781681731964