Human-Centric Interfaces for Ambient Intelligence

1.6 GESTURAL INFRASTRUCTURES

The gesture and speech infrastruct ure (GSI) [39] was built using DiamondTouch and

DViT technologies as described in Section 1.4. While it is a multimodal system

(speech and gesture) the gestures GSI supports are very simple: one-finger pointing

for selection, dragging, and pann ing; two-finger stretching for zooming in and out of

a region; fist-stamping to crea te new object s; and palm-down wiping to delete

objects. For example, a two-handed gesture might be multiple-object selection by

surrounding an area with upright hands, used in many multi touch applications for

grouping objects or as area selection. Sharepic includes a range of multihand and

multitouch gestures such as two-handed copy and two-person reset (wherein two

people can collaborate on a single two-handed (per person) gesture) [2].

RoomPlanner is furniture layout application developed with DiamondTouch tech-

nology [40]. It is designed for face-to-face work by two people sitting across a table

who have a 2D overhead view of the room and the furniture. The interaction techni-

ques here are more diverse than those in GSI or Sharepic. They are classified as follows:

n

Single-finger input techniques for tapping and dragging (to select and move

furniture in the room and to interact with the context-sensitive menu)

n

Two-finger input techniques for rotation and scaling

Table 1.3 Application Control Gestures

Gesture

Type

Graphics Action Description

Application

Control

Show

desktop

Touch spread thumb and three fingers. Slide

left.

Exit

application

Touch and rotate clockwise spread thumb

and three fingers.

Switch

application

Spread hand, then touch three fingers and

thumb and slide left or right. Slide crisply to

advance just one window, or gradually to

scroll through whole list.

Source: http://www.fingerworks.com.

24 CHAPTER 1 Face-to-Face Collaborative Interfaces

n

Single-hand techniques:

p

Flat hand on the table to rotate the display

p

Vertical hand to sweep the furniture pieces (as one would sweep objects

across a real table top)

p

Horizontal hand on the table to display the properties of objects found in

that area

p

Tilted horizontal hand with front projection to create a physical space where

private information can be projected

n

Two-handed techniques:

p

Both hands vertical, moving away from each other to spread furniture

around the room

p

Both hands vertical, moving toward each other to bring objects in to the area

delimited by them

p

Corner-shaped hands used to create a rectangular editing plane

1.6.1 Gestural Software Support

The NUI Snowflake system addresses the needs of third parties in the development

of interfaces for collaborative multitouch interaction. While supporting both FTIR

and DI, it uses computer vision and image processing to recognize motion, gestures,

and multitouch interaction. Figure 1.13 shows a multiuser, multi-input collaborative

photographic application running on an NUI Horizon device.

The DiamondSpin Toolkit [38] also supports third parties in the development of

interfaces. Along with support for DiamondTouch, its main feature is a real-time

polar-to-Cartesian transformation engine that enables around-the-table interactions.

Among the applications built on top of the toolkit is Table for N, designed for a small

number of people at a table creating, sketching, annotating, manipulating, and

browsing various types of documents, including text, html, images, and video clips.

Another application is UbiTable, which enables rapid display ecosystem formation

FIGURE 1.13

Collaborative photo-sharing application. Courtesy Natural User Interface Europe AB. Copyright

2008 Natural User Interface Europe AB. All rights reserved.

1.6 Gestural Infrastructures 25