Fast and Robust Mid-Air Gesture Typing for AR Headsets using 3D Trajectory Decoding.

Area of Science:

• Human-Computer Interaction
• Augmented Reality Interfaces
• Gesture Recognition

Background:

• Traditional augmented reality (AR) keyboards often rely on indirect 2D cursor control, which can be cumbersome.
• Mid-air gesture input for AR requires efficient and intuitive methods for text articulation.

Purpose of the Study:

• To develop and evaluate a fast mid-air gesture keyboard for head-mounted optical see-through augmented reality (OST AR).
• To introduce a novel decoding method for direct translation of 3D fingertip gestural trajectories into text.
• To assess the system's efficacy, learning acceleration, and performance without visual feedback.

Main Methods:

• Development of a novel decoding method for direct 3D fingertip trajectory to text translation.
• Evaluation through three user studies focusing on immediate efficacy, learning curves, and performance without visual feedback.
• Utilizing a mid-air gesture keyboard within an optical see-through augmented reality environment.

Main Results:

• The new 3D trajectory decoding design significantly improved text entry rates.
• The system demonstrated maintained low error rates during text input.
• Users sustained performance levels even without visual feedback of fingertip or gesture traces.

Conclusions:

• The proposed mid-air gesture keyboard offers an efficient input method for augmented reality.
• Direct 3D trajectory decoding enhances user performance in AR text entry.
• The system's robustness is highlighted by sustained performance without visual feedback, suggesting intuitive interaction.