Developing and testing a human-based gesture vocabulary for tabletop systems

The goal was to study the natural and intuitive use of surface gestures for the development of a tabletop system. Furthermore, the effect of expertise on choice of gestures was examined.

It is still not well understood what kinds of gestures novice users choose when they interact with gesture recognition systems.

First, novices' and experts' choice of gestures for a tabletop system was compared in a quasiexperimental design. Second, memorability of novices' and experts' gesture sets derived from the first study was compared in an experimental study. Third, memorization of hand shape and motion path was examined in a further experiment.

Data revealed user preferences for specific hand shapes and motion paths. Choice of gestures was affected by the size of the manipulated object,expertise, and nature of the command (direct manipulation of objects vs. assessment of abstract functions). Follow-up experiments revealed that the novices' gesture set was better memorized than were the experts' gesture set. Furthermore, the motion path of a gesture is better memorized than the specific hand shapes for a gesture.

Expertise affects the choice of gesture to a certain degree. It is therefore essential to involve novice users in the development of gesture vocabularies. Gestures for technical systems should be simple and should involve distinctive motion patterns instead of focusing on specific hand shapes or number of fingers. Abstract or symbolic gestures should be avoided.

Results of the study can be applied to the development of surface gestures for tabletop systems.