Entropy-Based Dual-Teacher Distillation for Efficient Motor Imagery EEG Classification.

Area of Science:

• Neuroscience
• Computer Science
• Signal Processing

Background:

• Motor imagery (MI) electroencephalography (EEG) classification is crucial for noninvasive brain-computer interfaces (BCIs).
• Ensembling improves MI decoding accuracy but incurs high inference costs, hindering low-latency applications.
• Small and noisy MI datasets present challenges like elevated predictive entropy and unstable convergence.

Purpose of the Study:

• To propose an entropy-based dual-teacher distillation framework for efficient MI EEG classification.
• To transfer knowledge from computationally expensive ensemble models to a single, deployable backbone.
• To address latency constraints in online or edge BCI deployments.

Main Methods:

• An entropy-based dual-teacher distillation framework was developed.
• An exponential moving average (EMA) teacher with entropy-gated activation was used to reduce student prediction noise.
• A two-stage cosine annealing schedule was employed to improve convergence robustness.

Main Results:

• The proposed method achieved consistent accuracy gains over ensemble teachers and distillation baselines on public MI benchmarks (BCI Competition IV-2a and IV-2b).
• On IV-2a, average accuracy reached 0.7713, outperforming original models (0.7222) and ensembles (0.7482).
• On IV-2b, average accuracy was 0.8583, exceeding original models (0.8432) and ensembles (0.8529).

Conclusions:

• The entropy-based dual-teacher distillation framework effectively transfers ensemble knowledge to a single backbone for MI EEG classification.
• This approach enhances decoding accuracy while satisfying strict latency requirements for BCI applications.
• The method demonstrates robustness and improved performance on challenging MI datasets.