Real-time detection and discrimination of visual perception using electrocorticographic signals

Area of Science:

• Neuroscience
• Computational Neuroscience
• Cognitive Science

Background:

• The ventral temporal cortex is known to contain specialized regions for visual stimulus processing.
• Electrocorticography (ECoG) offers high temporal and spatial resolution for studying neural activity.

Purpose of the Study:

• To investigate the spatial and temporal dynamics of ECoG responses to various visual stimuli types and colors.
• To develop and demonstrate a real-time decoder for identifying untrained natural images based on ECoG signals.

Main Methods:

• ECoG signals were recorded from four participants viewing diverse visual stimuli (faces, objects, bodies, digits, characters) in grayscale and color.
• A real-time system classified ECoG responses to faces, kanji, black screens, and natural scenes.
• Discrimination performance was assessed using broadband gamma (γ) activity.

Main Results:

• Offline classification achieved 72.9% accuracy for discriminating seven stimulus types and 67.1% for grayscale versus color images.
• A real-time decoder achieved 73.7% accuracy for basic stimuli and 74.8% for natural scenes.
• Stimulus categories were detected in real-time within 500 ms of stimulus onset.

Conclusions:

• Broadband γ activity in ECoG signals can detect and discriminate between different visual stimulus types and colors.
• Real-time decoding of visual stimuli is feasible, offering insights into neural processing dynamics.
• This study provides the first ECoG-based evidence for color-related population-level cortical broadband γ responses in humans.