Visual Scene Understanding for Enhanced EMG Gesture Recognition.

Area of Science:

• Biomedical Engineering
• Robotics
• Human-Computer Interaction

Background:

• Myoelectric prostheses rely on electromyography (EMG) for control, but are susceptible to noise and false activations.
• Real-world application requires robust gesture recognition that accounts for environmental context.
• Integrating multiple sensor modalities can improve the reliability and safety of prosthetic control systems.

Purpose of the Study:

• To develop and evaluate a multimodal framework for enhanced real-time gesture recognition in myoelectric prosthesis control.
• To augment EMG-based gesture recognition with computer vision (CV) for context-awareness.
• To mitigate false movements in prosthetic control by preventing erroneous gesture detection.

Main Methods:

• A multimodal approach combining electromyography (EMG) and computer vision (CV) was implemented.
• A Siamese deep convolution neural network (SDCNN) was used for EMG hand gesture recognition.
• A tailored YOLO computer vision model was employed for object detection to provide contextual information.
• Sensor fusion integrated SDCNN predictions with context from the YOLO model.

Main Results:

• The multimodal system demonstrated robust real-time gesture recognition in a real-world setting.
• Contextual information from CV effectively mitigated false gesture detection during onset and maintenance.
• Pilot experiments confirmed enhanced robustness of the gesture control interface.
• Users maintained better command over the prosthetic system due to improved control accuracy.

Conclusions:

• Multimodal sensor fusion of EMG and CV offers a promising approach for reliable myoelectric prosthesis control.
• Context-aware frameworks significantly improve the safety and usability of advanced prosthetic devices.
• This integrated system enhances human volitional control by increasing interface robustness and user command.