Brain-Machine Coupled Learning Method for Facial Emotion Recognition.

Area of Science:

• Artificial Intelligence
• Neuroscience
• Computer Vision

Background:

• Machine learning models for facial emotion recognition (FER) struggle with limited generalization from small datasets.
• Human brains effectively process visual information and generalize from few samples, a capability lacking in current AI.
• Bridging the gap between machine visual processing and human cognitive understanding is crucial for advancing FER.

Purpose of the Study:

• To propose a novel brain-machine coupled learning method for facial emotion recognition (FER).
• To enable neural networks to simultaneously acquire machine visual knowledge and human cognitive knowledge.
• To improve the generalization ability of FER models by leveraging brain-inspired learning.

Main Methods:

• Utilized both visual images and electroencephalogram (EEG) signals for coupled training in visual and cognitive domains.
• Developed domain-specific models with interactive common and private channels.
• Employed reverse engineering to decode cognitive processes from EEG signals and adversarial strategies for domain-specific knowledge extraction.

Main Results:

• The common channel in the visual domain successfully approximated cognitive processes decoded from EEG signals.
• Private channels captured domain-specific knowledge through adversarial learning.
• The final model, using only visual domain channels, achieved excellent performance on public datasets without EEG input.

Conclusions:

• The proposed brain-machine coupled learning method significantly enhances facial emotion recognition performance.
• The method demonstrates strong generalization capabilities on new datasets, even when trained with EEG signals.
• This approach holds potential for practical applications in FER and can be adapted to other neural network models.