A Hierarchical Hand Gesture Recognition Framework for Sports Referee Training-Based EMG and Accelerometer Sensors

Area of Science:

• Sports Science
• Biomedical Engineering
• Machine Learning

Background:

• Current sensor-based methods struggle to recognize diverse hand gestures for sports referee training.
• Existing techniques using handcrafted features are insufficient for both large and subtle motion gestures.
• Accurate recognition of official referee signals (ORSs) and interactive gestures is crucial for effective training.

Purpose of the Study:

• To develop an advanced sports referee training system capable of recognizing trainee gestures.
• To improve hand gesture recognition accuracy by integrating deep belief networks (DBNs) with handcrafted features.
• To create a hierarchical recognition scheme for distinguishing between large and subtle motion gestures.

Main Methods:

• Utilized deep belief networks (DBNs) for learning representative features from sEMG and IMU data.
• Combined DBN-derived features with selective handcrafted features for robust gesture recognition.
• Implemented a hierarchical recognition strategy classifying gestures as large or subtle motion before final recognition.

Main Results:

• The proposed hierarchical scheme integrating DBN features from multimodal data achieved superior recognition accuracy.
• Fusion of heterogeneous signals from the Myo armband (sEMG and IMU) enhanced overall recognition performance.
• The system demonstrated effectiveness in recognizing both official referee signals and interactive gestures in basketball training.

Conclusions:

• The developed sports referee training system, leveraging DBNs and a hierarchical approach, significantly improves gesture recognition.
• Multimodal data fusion and advanced feature learning offer a robust solution for complex gesture recognition tasks.
• This system provides a promising tool for cultivating professional sports referees through accurate performance feedback.