Multi-Scale Feature Extraction and Aggregation Network for Electroencephalography Classification in Face Photo-Sketch...

Area of Science:

• Cognitive Neuroscience
• Computer Vision
• Biometrics

Background:

• Face photo-sketch recognition is vital for forensics and biometrics.
• Significant modality and semantic gaps challenge accurate recognition.
• Existing methods struggle to bridge the gap between diverse facial representations.

Purpose of the Study:

• To propose an effective electroencephalography (EEG)-based approach to bridge the modality gap in face photo-sketch recognition.
• To introduce a novel EEG signal feature decoding method, the multi-scale feature extraction and aggregation network (MFEA).
• To evaluate the performance of MFEA using EEG data from a rapid serial visual presentation (RSVP) paradigm.

Main Methods:

• Developed a face photo-sketch recognition paradigm (FPSR) using rapid serial visual presentation (RSVP).
• Proposed the multi-scale feature extraction and aggregation network (MFEA) for decoding EEG signals.
• MFEA extracts and aggregates multi-scale shallow and deep EEG features, followed by spatial dimensionality reduction.

Main Results:

• Experimental evaluation on public and self-conducted EEG RSVP datasets.
• MFEA demonstrated superior performance in EEG classification compared to previous methods.
• The proposed method effectively enhances the retention of relevant EEG signal features.

Conclusions:

• The MFEA method offers a significant advancement in EEG-based face photo-sketch recognition.
• This approach effectively addresses the challenges posed by the modality and semantic gaps.
• The findings have implications for improving forensic investigations and facial biometrics.