Word-Level Motion Learning for Contactless QWERTY Typing with a Single Camera.

Area of Science:

• Computer Vision
• Human-Computer Interaction
• Machine Learning

Background:

• Vision-based text entry is crucial for immersive and constrained environments.
• Current methods often rely on fragile character-level recognition or key localization with single cameras.
• Monocular sensing presents challenges for accurate contactless text input.

Purpose of the Study:

• To investigate the feasibility of word-level contactless typing recognition using a single RGB camera.
• To develop a framework for recognizing typing motions directly as word-level patterns.
• To assess the robustness and adaptability of the proposed method.

Main Methods:

• A word-level contactless typing framework was developed, modeling words as spatiotemporal finger motion patterns from hand joint trajectories.
• Typing motions were temporally segmented, and direction-aware finger displacements were accumulated for motion representation.
• Online learning with a trial-delayed adaptation protocol was used to update word motion prototypes.

Main Results:

• Experiments with up to 200 words demonstrated progressive learning and recall of word-level motion patterns.
• Stable recognition performance was achieved at realistic typing speeds within the tested single-user, single-camera setup.
• Learned motion representations transferred from physical keyboards to flat-surface typing, even with reduced tactile and visual cues.

Conclusions:

• Contactless typing can be effectively reframed as a word-level motion recall problem.
• The proposed method shows potential as a complementary input method for constrained monocular sensing.
• Word-level motion pattern recognition offers a robust alternative to character-centric approaches in specific scenarios.