A neurophysiologically interpretable deep neural network predicts complex movement components from brain activity | JoVE Visualize

Area of Science:

Neuroscience
Machine Learning
Biomedical Engineering

Background:

Effective decoding of electroencephalography (EEG) is crucial for neurorehabilitation and neural prosthetics.
Current deep learning models for EEG face limitations due to coarse motor task datasets and weak neurophysiological interpretability, hindering generalizability.
There is a need for advanced neural network architectures that capture spatiotemporal dependencies and align with brain activity during movement.

Purpose of the Study:

To develop and validate a neurophysiologically interpretable 3-dimensional convolutional neural network (3D-CNN) for decoding complex hand movements from EEG.
To improve the accuracy and generalizability of EEG-based movement decoding by incorporating biological relevance.
To identify key EEG sensors and time segments critical for accurate movement classification.

Main Methods:

A novel 3D-CNN architecture was designed to process topography-preserving EEG inputs, capturing spatiotemporal brain activity.
The model was trained and validated on a new dataset from a motor experiment involving hand movements performed on a plane using a rehabilitation robot.
Classification tasks included reaction time (slow/fast), movement mode (active/passive), and movement direction (left/right/up/down).

Main Results:

The 3D-CNN achieved high average leave-one-subject-out test accuracies: 79.81% for reaction time, 81.23% for active vs. passive, and 82.00% for direction.
The proposed 3D-CNN outperformed a modern 2D-CNN architecture by 1.1% to 6.74% across different classification tasks.
Analysis revealed crucial EEG sensors and time segments, aligning with known neurophysiology of motor planning and execution.

Conclusions:

Neurophysiologically interpretable models are vital for accurate EEG decoding and enhance the generalizability of brain-computer interfaces.
The developed 3D-CNN demonstrates the potential for real-time classification of complex brain activities beyond motor tasks.
This research paves the way for more sophisticated and biologically relevant neural network applications in neuroscience and neurotechnology.