Person identification through faces and voices: an ERP study

Area of Science:

• Neuroscience
• Cognitive Science
• Psychology

Background:

• Models of identity extraction from faces and voices exist, but the temporal dynamics of their interaction are not well understood.
• Neuroimaging studies have explored neural sites of audio-visual interactions, yet precise timing remains elusive.

Purpose of the Study:

• To investigate the temporal dynamics of audio-visual interactions in person identity processing using electrophysiology.
• To determine when face and voice processing modules begin to interact during familiarity detection.

Main Methods:

• Event-related potentials (ERPs) were recorded while participants performed a familiarity detection task.
• Stimuli included faces alone (F), voices alone (V), and faces and voices presented together (FV).
• Responses to FV were compared to the summed responses of F + V to identify cross-modal interactions.

Main Results:

• Cross-modal interactions between face and voice processing were observed, indicated by differences in ERP amplitudes at specific latencies.
• For unfamiliar stimuli, interaction began early (around 200ms) and was confined to the N170 component time window.
• For familiar stimuli, interaction started earlier, lasted longer, and extended beyond the N170 component time window.

Conclusions:

• Audio-visual interactions in person recognition commence early in the processing stream and persist as familiarity assessment progresses.
• This study provides the first electrophysiological evidence for cross-modal interaction between faces and voices during the recognition of familiar individuals.
• Familiarity assessment may facilitate direct information sharing between voice and face sensory modules even before accessing person identity information.