img2fmri: a python package for predicting group-level fMRI responses to visual stimuli using deep neural networks.

Area of Science:

• Neuroscience
• Computer Science
• Machine Learning

Background:

• Deep neural networks (DNNs) excel at visual categorization tasks.
• DNNs have shown success in predicting cortical responses in the human visual cortex.
• Predicting brain activity from visual stimuli is a key challenge in neuroscience.

Purpose of the Study:

• Introduce img2fmri, a Python package for predicting group-level fMRI responses to individual images.
• Validate the prediction model's accuracy on unseen image datasets.
• Extend the model to predict fMRI responses to continuous visual stimuli, such as animated films.

Main Methods:

• Utilized an artificial deep neural network (DNN) for image-to-fMRI response prediction.
• Trained the DNN on visual categorization tasks.
• Validated the model by predicting fMRI responses to novel images and a short animated film.
• Analyzed timepoint-to-timepoint similarity of predicted fMRI responses around event boundaries.

Main Results:

• The img2fmri model accurately predicted fMRI responses to novel images.
• The frame-by-frame prediction model successfully predicted fMRI responses to a continuous visual stimulus (animated film).
• The model outperformed a baseline in describing real fMRI response dynamics around event boundaries, especially immediately before and at events.

Conclusions:

• The img2fmri package provides a novel tool for predicting brain activity from visual input.
• The study demonstrates that temporal dynamics in visual cortex processing of naturalistic stimuli can be explained by stimulus dynamics.
• The findings suggest DNNs are powerful tools for understanding brain responses to complex visual information.