Continual Learning of a Transformer-Based Deep Learning Classifier Using an Initial Model from Action Observation EEG...

Area of Science:

• Neuroscience
• Human-Computer Interaction
• Biomedical Engineering

Background:

• Motor imagery (MI) brain-computer interfaces (BCI) offer intuitive mind-controlled communication.
• Current MI-BCI systems suffer from high inter-subject variability due to unclear instructions, hindering big-data training.
• Virtual reality (VR) presents a novel environment for enhanced BCI training.

Purpose of the Study:

• To develop and evaluate a novel BCI training method using action observation concurrently with motor imagery (AO + MI) in a VR environment.
• To investigate the effectiveness of continuous learning through motor imagery with visual feedback (MI-FB) in improving BCI performance.
• To assess the performance of a transformer-based spatial-temporal network (TSTN) for decoding MI intentions.

Main Methods:

• Five healthy subjects participated in AO + MI, MI, and MI-FB tasks within a VR environment using a head-mounted device (HMD).
• EEG signals were recorded during tasks to train an initial model, which was continually improved with subsequent BCI training sessions.
• A transformer-based spatial-temporal network (TSTN) was employed for decoding MI intentions, focusing on spatial and temporal feature extraction with attention mechanisms.

Main Results:

• The AO + MI approach facilitated easier conformity of imagery actions for subjects.
• Mean detection accuracies using TSTN improved across MI-FB sessions, reaching 0.77 in the third session.
• BCI performance demonstrated significant improvement through the continual learning process of MI-FB training.

Conclusions:

• The proposed VR-based AO + MI training method effectively enhances user engagement and reduces variability in MI-BCI.
• Continuous learning via MI-FB significantly boosts BCI performance, demonstrating the efficacy of adaptive training strategies.
• The TSTN model shows promise for accurate decoding of MI intentions in complex BCI paradigms.