Identifying the Nature of Grip Force Signals in EEG & fNIRS with Multi-Modal Graph Fusion Network

Area of Science:

• Neuroscience
• Rehabilitation Engineering
• Biomedical Engineering

Background:

• Brain-Computer interfaces (BCIs) offer potential for motor rehabilitation in paralysis.
• Continuous force control is vital for natural movement but challenging to decode.
• Understanding brain coordination of motor commands and sensory feedback is crucial.

Purpose of the Study:

• Investigate brain coordination during force control.
• Develop a multi-modal model for continuous grip force decoding.
• Enhance BCIs for rehabilitation and assistive devices.

Main Methods:

• Novel experimental setup isolating motor intention and sensory feedback.
• Functional electrical stimulation for passive gripping.
• Multi-modal brain signal collection (EEG, fNIRS).
• Multi-modal graph fusion model for force decoding.

Main Results:

• Significant neural pattern differences observed in EEG time-frequency representations across conditions (voluntary movement, motor imagery, passive perception).
• Successful continuous bimanual grip force decoding using the fused EEG and fNIRS model.
• Demonstrated feasibility of combining motor intention and sensory feedback.

Conclusions:

• The study advances understanding of neural control during force tasks.
• The developed multi-modal BCI model shows promise for accurate force decoding.
• Findings support the development of advanced neuro-rehabilitation and assistive technologies involving force manipulation.