Image-Evoked Emotion Recognition for Hearing-Impaired Subjects with EEG Signals

Area of Science:

• Neuroscience
• Artificial Intelligence
• Biomedical Engineering

Background:

• Emotion recognition via electroencephalogram (EEG) signals is gaining traction.
• Hearing-impaired individuals may exhibit distinct information processing biases.
• Understanding emotional responses in this population is crucial for communication.

Purpose of the Study:

• To develop and evaluate an advanced EEG-based emotion recognition model for hearing-impaired individuals.
• To compare the model's performance between hearing-impaired and non-hearing-impaired subjects.
• To investigate brain topography differences in emotion processing.

Main Methods:

• Collected EEG data from hearing-impaired and non-hearing-impaired participants viewing emotional faces.
• Extracted spatial features using symmetry difference and quotient from original signals and differential entropy (DE).
• Proposed a multi-axis self-attention classification model integrating local and global attention with convolution.

Main Results:

• The proposed multi-axis self-attention model outperformed original feature methods.
• Multi-feature fusion demonstrated effectiveness across both subject groups.
• Achieved average accuracies of 70.2% (3-class) and 50.15% (5-class) for hearing-impaired, and 72.05% (3-class) and 51.53% (5-class) for non-hearing-impaired subjects.
• Identified distinct discriminative brain regions, including the parietal lobe, in hearing-impaired subjects.

Conclusions:

• The novel self-attention model significantly improves EEG-based emotion recognition, particularly for hearing-impaired individuals.
• Multi-feature fusion enhances classification accuracy.
• Brain topography analysis reveals unique emotional processing patterns in the hearing-impaired population.