GaborNet Visual Encoding: A Lightweight Region-Based Visual Encoding Model With Good Expressiveness and Biological...

Area of Science:

• Neuroscience
• Computational Neuroscience
• Machine Learning

Background:

• Computational visual encoding models are crucial for understanding neuronal activity in the brain's visual cortex.
• Challenges exist in creating non-linear feature spaces to accurately model neuronal responses.

Purpose of the Study:

• To introduce GaborNet-VE, a novel end-to-end visual encoding model for the ventral stream.
• To address limitations in current models by incorporating learnable Gabor kernels.

Main Methods:

• The GaborNet-VE model utilizes a Gabor convolutional layer, two regular convolutional layers, and a fully connected layer.
• It employs learnable Gabor kernels in the initial layer, replacing standard convolutional kernels.
• The model efficiently encodes all voxels within a region of interest from functional magnetic resonance imaging (fMRI) data.

Main Results:

• GaborNet-VE achieved state-of-the-art prediction performance in the primary visual cortex.
• Visualizations confirmed the model's ability to fit visual features and estimate receptive fields accurately.
• The model demonstrated strong expressiveness and biological interpretability.

Conclusions:

• The proposed GaborNet-VE model offers a lightweight and effective approach to visual encoding.
• Combining handcrafted (Gabor filters) and deep learning features enhances model performance and interpretability.
• This model advances the understanding of stimulus-response relationships in the visual cortex.