The P3 indexes the distance between perceived and target image

Area of Science:

• Cognitive Neuroscience
• Computer Vision
• Machine Learning

Background:

• Visual recognition involves comparing perceived objects to mental targets, a challenge with complex stimuli like faces.
• Previous research linked visual similarity to P300 amplitude, but lacked a precise measure for complex stimuli.
• Generative adversarial networks (GANs) offer a novel approach to modeling complex visual relationships.

Purpose of the Study:

• To redefine visual similarity using latent space distances derived from GANs.
• To investigate the relationship between GAN-derived image distances and the P300 brainwave amplitude.
• To explore how GANs can model perception and recognition of complex visual stimuli.

Main Methods:

• Utilized a state-of-the-art generative adversarial network (GAN) to learn a latent space representation of images.
• Conducted a rapid serial visual presentation experiment using oddball images at varying GAN-derived distances from a target.
• Measured P300 amplitude in response to these varying distances to assess perceptual identification.

Main Results:

• A monotonic relationship was found between the GAN-derived distance-to-target and P300 amplitude.
• Perceptual identification was associated with smooth, continuous changes in image similarity.
• P300 sub-components (P3a and P3b) showed distinct but similarly related responses to target distance.

Conclusions:

• The P300 brainwave effectively indexes the distance between perceived and target images in complex visual stimuli.
• GANs provide a powerful new methodology for studying the relationship between stimuli, perception, and recognition.
• This study demonstrates a smooth, drifting similarity measure for complex visual perception.