Variational autoencoder: An unsupervised model for encoding and decoding fMRI activity in visual cortex.

Area of Science:

• Neuroscience
• Computational Neuroscience
• Machine Learning

Background:

• Deep neural networks like convolutional neural networks (CNNs) are used to model visual cortex responses.
• Variational auto-encoders (VAEs) present an alternative deep learning architecture for visual representation learning.

Purpose of the Study:

• To explore VAEs as computational models of the human visual cortex.
• To compare VAE performance against CNNs in predicting and decoding brain activity related to visual stimuli.

Main Methods:

• Trained a five-layer encoder-decoder VAE on unlabeled images.
• Predicted and decoded functional magnetic resonance imaging (fMRI) data from human subjects viewing videos.
• Compared VAE performance with CNNs and partial least squares regression.

Main Results:

• VAE achieved comparable prediction accuracy to CNNs in early visual areas, but lower accuracy in higher-order areas.
• VAEs provided a more convenient decoding strategy for reconstructing visual input from fMRI data.
• VAE-based reconstruction preserved spatial structure and color, outperforming other decoding methods.

Conclusions:

• VAEs serve as effective unsupervised models for learning visual representations.
• VAEs demonstrate potential for explaining visual cortical responses and reconstructing visual experiences.
• The learning objective, not architecture, differentiates VAE and CNN encoding performance.