A multi-level teacher assistant-based knowledge distillation framework with dynamic feedback for motor imagery EEG...

Area of Science:

• Neuroscience
• Machine Learning
• Biomedical Engineering

Background:

• Deep learning models show potential for motor imagery-based electroencephalogram (MI-EEG) decoding in non-invasive brain-computer interfaces (BCIs).
• Computational demands of deep learning hinder practical BCI deployment, leading to the exploration of knowledge distillation (KD) for model compression.
• Existing KD methods face challenges in effectively transferring multi-level MI-EEG signal knowledge under high compression.

Purpose of the Study:

• To propose a novel knowledge distillation framework, Motor Imagery Knowledge Distillation (MIKD), for compressing deep learning models in MI classification tasks.
• To enhance the transfer of multi-level knowledge from complex teacher models to smaller student models for MI-EEG decoding.
• To maintain high classification performance despite significant model compression.

Main Methods:

• Development of the MIKD framework featuring a multi-level teacher assistant knowledge distillation (ML-TAKD) module.
• ML-TAKD module designed to extract and transfer local representations and global dependencies from MI-EEG signals.
• Integration of a dynamic feedback module for adaptive teaching strategies based on student learning progress.

Main Results:

• MIKD framework achieved state-of-the-art performance across three public EEG datasets.
• Significant accuracy improvements observed: 6.61%, 1.91%, and 3.29% on the respective datasets compared to the baseline student model.
• Achieved substantial model size reduction of nearly 90%.

Conclusions:

• The MIKD framework offers an effective solution for compressing deep learning models for MI-EEG decoding.
• MIKD successfully addresses the limitations of vanilla KD methods in transferring complex MI-EEG signal knowledge.
• The proposed method enables practical deployment of efficient and high-performing deep learning models in BCI applications.