Convolutional neural network-based encoding and decoding of visual object recognition in space and time.

Area of Science:

• Neuroscience
• Computer Vision
• Cognitive Science

Background:

• Deep convolutional neural networks (CNNs) are increasingly used to model the human visual system.
• Previous studies linked CNNs to brain activity using fMRI, but high temporal resolution data (like MEG) is less explored.
• Understanding visual processing requires examining both spatial and temporal dynamics.

Purpose of the Study:

• To investigate if CNN representations correspond to brain activity at high temporal resolution.
• To model human cortical activity using CNNs and magnetoencephalography (MEG).
• To explore the feed-forward sweep of visual information processing.

Main Methods:

• Used CNN-based encoding models with human magnetoencephalography (MEG) data.
• Acquired MEG signals while participants viewed 1,000 object images.
• Modeled source-reconstructed cortical activity using CNN representations.

Main Results:

• Observed a feed-forward sweep of activity across the visual hierarchy from 75 to 200 ms post-stimulus onset.
• CNN layer representations mirrored the spatiotemporal cascade of visual information.
• Stimulus representations in the CNN model mapped to distinct visual cortex regions, following the ventral stream.

Conclusions:

• CNNs can effectively model high-temporal-resolution human brain activity related to visual object recognition.
• The findings support the hierarchical processing of visual information along the ventral stream.
• The study achieved state-of-the-art decoding accuracy for object identity.