Gesture centric interaction: evaluating hand and head gestures in touchless cursor control

Area of Science:

• Human-Computer Interaction
• Computer Vision
• Biomedical Engineering

Background:

• Touchless interfaces are increasingly important for user-friendly and hygienic interaction.
• Advancements in computer vision enable gesture and movement recognition for control systems.

Purpose of the Study:

• To design and evaluate two touchless cursor control systems: one based on hand gestures and another on head movements.
• To compare the performance and user experience of these two distinct touchless control modalities.

Main Methods:

• Utilized the MediaPipe framework with a laptop camera to extract hand and face landmarks.
• Developed a hand gesture system tracking index finger landmarks for cursor control.
• Implemented a head movement system calculating yaw and pitch angles for cursor control.

Main Results:

• Hand gesture control achieved a higher throughput (0.59 bps) compared to head movement control (0.53 bps) in a 2D Fitts' law experiment.
• Participants reported a better overall experience and perceived lower task difficulty with the hand gesture system.

Conclusions:

• Touchless cursor control using hand gestures offers superior performance and user satisfaction compared to head movement-based control.
• The MediaPipe framework provides an effective tool for developing such gesture and movement-based human-computer interfaces.