Decoding ECoG signal into 3D hand translation using deep learning.

Area of Science:

• Neuroscience and Biomedical Engineering
• Brain-Computer Interfaces (BCIs)
• Machine Learning in Healthcare

Background:

• Motor brain-computer interfaces (BCIs) offer a promising avenue for individuals with tetraplegia to regain environmental interaction capabilities, particularly arm and hand function.
• Electrocorticography (ECoG)-based BCIs provide a balance between invasiveness and signal resolution, but current linear decoders struggle to capture complex relationships for continuous movement prediction.
• Deep learning (DL) models present a potential solution to overcome the limitations of linear models in decoding ECoG signals for precise motor control.

Purpose of the Study:

• To evaluate the efficacy of various deep learning (DL) architectures for predicting imagined 3D continuous hand translation from electrocorticography (ECoG) signals.
• To compare the performance of DL-based decoders against traditional multilinear models in a real-world BCI application for a tetraplegic subject.

Main Methods:

• Time-frequency features were extracted from ECoG signals recorded during a closed-loop experiment with a tetraplegic participant.
• Several DL architectures, including multilayer perceptron, convolutional neural networks (CNNs), and long short-term memory (LSTM) networks, were implemented and tested.
• The accuracy of DL models was compared offline to a multilinear model using cosine similarity metrics.

Main Results:

• CNN-based DL architectures demonstrated superior performance compared to the state-of-the-art multilinear model.
• The optimal architecture integrated CNNs to leverage spatial correlations between electrodes and LSTMs to capture the temporal dynamics of hand trajectories.
• Deep learning models achieved significant improvements in prediction accuracy, increasing average cosine similarity by up to 60% for both left (0.189 to 0.302) and right (0.157 to 0.249) hands.

Conclusions:

• Deep learning models offer a substantial advancement in decoding ECoG signals for predicting 3D hand movements in individuals with tetraplegia.
• The proposed DL approaches, particularly those combining CNN and LSTM, enhance the accuracy and potential utility of BCIs for restoring motor function.
• This study highlights the potential of DL to improve the performance and real-world applicability of brain-computer interface systems.