A Multimodal Bracelet to Acquire Muscular Activity and Gyroscopic Data to Study Sensor Fusion for Intent Detection.

Area of Science:

• Biomedical Engineering
• Robotics
• Human-Computer Interaction

Background:

• Controlling robotic hands and prostheses with biosignals remains a challenge, with surface electromyography (sEMG) being the primary but limited method.
• Existing sEMG-based systems struggle to meet reliability requirements for practical use in hand motion decoding.
• Combining multiple sensor types can enhance hand gesture classification accuracy.

Purpose of the Study:

• To develop and evaluate a novel multimodal bracelet for improved hand gesture classification.
• To investigate the efficacy of sensor fusion for intent detection in advanced prosthetic and robotic hand control.
• To assess the performance of a combined force myography (fMG) and sEMG system with inertial measurement units (IMUs).

Main Methods:

• A multimodal bracelet was designed, integrating a 24-channel fMG system with six commercial sEMG sensors, each equipped with a six-axis IMU.
• Five participants performed five distinct gestures, with muscular activity recorded using the proposed device.
• A random forest model was employed to classify gestures based on the acquired multimodal sensor data.

Main Results:

• The multimodal device demonstrated functionality for acquiring muscular activity for gesture classification.
• Combining all sensor modalities (fMG, sEMG, IMU) achieved the highest average classification accuracy of 92.3±2.6%.
• Sensor fusion reduced misclassifications by 37% compared to sEMG alone and 22% compared to fMG alone.

Conclusions:

• The developed multimodal bracelet is suitable for studying sensor fusion in intent detection.
• Sensor fusion offers significant benefits for robust and accurate hand gesture classification.
• This approach shows promise for advancing the control of robotic and prosthetic hands.