LLM-Enhanced Multi-Teacher Knowledge Distillation for Modality-Incomplete Emotion Recognition in Daily Healthcare.

Area of Science:

• Neuroscience
• Artificial Intelligence
• Machine Learning

Background:

• Monitoring human emotional states is crucial in healthcare.
• Electroencephalogram (EEG)-based emotion recognition faces challenges with incomplete data due to acquisition constraints.
• Existing knowledge distillation methods struggle with transferring integrated spatial and temporal EEG features.

Purpose of the Study:

• To propose a multi-teacher knowledge distillation framework to address incomplete modalities in EEG-based emotion recognition.
• To leverage a Large Language Model (LLM) for enhanced feature learning in student networks.
• To improve the transfer of integrated spatial and temporal features from multimodal to unimodal EEG analysis.

Main Methods:

• Developed a multi-teacher knowledge distillation framework incorporating an LLM and a graph convolutional neural network.
• Utilized the LLM as a teacher for temporal feature extraction and a GCN for spatial feature extraction.
• Integrated causal masking and a confidence indicator into the LLM to refine feature transfer.

Main Results:

• The proposed framework significantly outperformed existing methods in modality-incomplete scenarios.
• Demonstrated superior performance on the DEAP and MAHNOB-HCI datasets.
• Validated the effectiveness of LLM-enhanced knowledge distillation for EEG emotion recognition.

Conclusions:

• The multi-teacher knowledge distillation framework effectively handles incomplete EEG data for emotion recognition.
• Large Language Models show significant potential in advancing EEG-based affective computing.
• The study provides a robust solution for practical healthcare applications requiring reliable emotion state monitoring.