Dual-Modal Gesture Recognition Using Adaptive Weight Hierarchical Soft Voting Mechanism

Area of Science:

• Biomedical Engineering
• Human-Computer Interaction
• Signal Processing

Background:

• Gesture recognition relies on understanding muscle activity.
• Surface Electromyography (sEMG) captures muscle electrophysiological signals.
• A-mode ultrasound (AUS) provides muscle morphological data.

Purpose of the Study:

• To develop a robust gesture recognition method by fusing sEMG and AUS data.
• To introduce the adaptive weight classification (AWC) and adaptive weight hierarchical soft voting (AWHSV) modules.
• To enhance information representation and algorithm robustness in gesture recognition.

Main Methods:

• Integration of sEMG and AUS data into a fused modality.
• Development of the adaptive weight classification (AWC) module.
• Implementation of adaptive weight hierarchical soft voting (AWHSV) with hierarchical classifiers.

Main Results:

• The AWHSV method achieved higher recognition rates compared to using sEMG or AUS alone.
• The fused sEMG-AUS modality with AWHSV outperformed individual modalities by 0.66% to 2.36%.
• The proposed method demonstrated superior performance against state-of-the-art approaches in gesture recognition.

Conclusions:

• The AWHSV method effectively fuses sEMG and AUS data for enhanced gesture recognition.
• Dynamic weight adjustment in AWHSV compensates for information loss during fusion.
• This approach offers broader application scenarios for robust gesture recognition.