Comparing visual representations across human fMRI and computational vision.

Area of Science:

• Neuroscience
• Computer Vision
• Cognitive Science

Background:

• Visual object perception is thought to follow a hierarchical cortical pathway.
• The intermediate features driving this object representation transformation are not well understood.

Purpose of the Study:

• To investigate how well different computer vision models explain neural object encoding in the human visual cortex.
• To identify which computational models best capture intermediate visual object representations.

Main Methods:

• fMRI data were collected while participants viewed 60 real-world object images.
• Searchlight analysis and representational dissimilarity analysis were used to compare neural activity patterns with computer vision model outputs.
• Multiple computer vision algorithms were tested for their ability to account for neural data.

Main Results:

• All tested computer vision models showed some ability to account for neural data.
• The Scale Invariant Feature Transform (SIFT) model best explained neural representations in the ventral visual pathway.
• Significant matches between models and neural data were found in ventral-temporal cortex regions associated with intermediate object perception.

Conclusions:

• Computer vision models, especially those encoding local features like SIFT, can serve as effective proxies for understanding intermediate visual object representations.
• Differences in model performance suggest that the featural basis (holistic vs. parts-based) influences how well models capture neural representations.