Neural correlates of face perception modeled with a convolutional recurrent neural network.

Area of Science:

• Computational neuroscience
• Visual neuroscience
• Machine learning in neuroimaging

Background:

• Event-related potentials (ERPs) show distinct sensitivity to faces, characterized by the N170 peak.
• The N170 exhibits greater amplitude and shorter latency for human faces compared to other objects.
• Understanding the neurophysiological basis of this face sensitivity is crucial for visual neuroscience.

Purpose of the Study:

• To develop a computational model for generating visual event-related potentials (ERPs).
• To investigate the phenomenon of face-sensitive ERPs using a deep learning approach.
• To model the relationship between visual stimuli and evoked neural activity.

Main Methods:

• Developed a hybrid model combining a 3D convolutional neural network (CNN) for image representation and a recurrent neural network (RNN) for sequence learning.
• Utilized open-access ERP data and generated synthetic images with a generative adversarial network for training and validation.
• Represented visual stimuli as image sequences (time x pixels) and trained the model end-to-end to predict ERP waveforms.

Main Results:

• The computational model demonstrated strong correlations with both open-access (r=0.98) and validation (r=0.78) datasets.
• Model performance showed consistency with some aspects of neural recordings, while highlighting limitations in capturing all neurophysiological details.
• The model successfully reproduced ERP waveforms evoked by visual events.

Conclusions:

• The developed CNN-RNN model shows significant promise for studying visual ERP generation and face sensitivity.
• This computational approach can be adapted to explore various aspects of visual neuroscience and neural activity.
• The model offers a valuable tool for understanding the computational underpinnings of visual perception.