Generalized Cross-Domain Framework for Gesture Recognition via Wrist-Worn Sensing.

Area of Science:

• Human-Computer Interaction
• Wearable Sensing Technology
• Machine Learning for Gesture Recognition

Background:

• Domain shifts in wrist-worn sensors challenge cross-domain gesture recognition.
• Natural and convenient human-computer interaction relies on effective gesture recognition.

Purpose of the Study:

• To propose a generalized cross-domain framework for fine-grained gesture recognition using single-site wrist-worn sensing.
• To investigate optimal fine-tuning strategies and analyze cross-domain mechanisms.
• To determine the trade-off between accuracy and computational cost in gesture recognition.

Main Methods:

• Developed a Multi-Branch Network incorporating feature-level multimodal fusion and inter-modal interaction.
• Constructed a multimodal dataset with 15 static and 18 dynamic gestures.
• Evaluated five fine-tuning strategies across cross-session, cross-subject, cross-gesture, and cross-modality paradigms.

Main Results:

• Optimal transfer learning ratios for cross-domain paradigms ranged from 6.1% to 9.0%, clustering around 9.0%.
• Identified valuable insights into selecting optimal fine-tuning strategies and understanding cross-domain mechanisms.
• Demonstrated the feasibility of a real-time online gesture recognition system in real-world scenarios.

Conclusions:

• The proposed framework and Multi-Branch Network effectively address domain shifts for fine-grained gesture recognition.
• This research provides a reference for optimal transfer learning ratios in diverse cross-domain scenarios.
• The study validates the potential of wrist-worn single-site sensing for advanced gesture recognition applications.