Visual dictionaries as intermediate features in the human brain

Area of Science:

• Computational neuroscience
• Computer vision
• Human visual system

Background:

• The human visual system processes low-level features into complex object representations.
• Computational models like HMAX and Bag of Words (BoW) use intermediate features for scene representation.
• Understanding the brain's computational mechanisms for intermediate features remains a challenge.

Purpose of the Study:

• To compare the HMAX and Bag of Words (BoW) models in explaining human fMRI responses.
• To investigate how hierarchical levels of these models account for brain activity during visual perception.
• To determine which model more accurately reflects neural processing of intermediate visual features.

Main Methods:

• Functional magnetic resonance imaging (fMRI) data from 20 subjects viewing a video clip.
• Voxel-wise analysis using distance-based variation partitioning.
• Comparison of HMAX and BoW model predictions against brain activity across visual hierarchy.

Main Results:

• Both HMAX and BoW models significantly explain brain activity in early visual areas (V1, V2, V3).
• The BoW model demonstrated greater consistency across subjects in explaining visual cortex activity than HMAX.
• Both models accounted for some brain activity in higher visual areas associated with intermediate feature processing.

Conclusions:

• The Bag of Words (BoW) model appears to more faithfully represent neural responses in the human visual system, particularly in low and intermediate visual areas.
• Computational models offer valuable insights into the brain's visual processing mechanisms.
• Further research can refine these models to better understand visual cognition.