Optimized AI-based neural decoding from BOLD fMRI signal for analyzing visual and semantic ROIs in the human visual...

Area of Science:

• Neuroscience
• Artificial Intelligence
• Computer Vision

Background:

• AI-driven neural decoding aims to reconstruct visual perception from fMRI brain activity using generative models.
• Current methods often involve complex, multi-stage processes with variational autoencoders (VAE) and latent diffusion models (LDM).
• Challenges include fMRI data complexity, noise, and optimizing the interplay between reconstruction stages.

Purpose of the Study:

• To develop and evaluate an optimized two-stage AI architecture for neural decoding of visual stimuli from fMRI data.
• To address gaps in existing two-stage models by implementing non-linear mappings, optimizing latent space dimensionality, and analyzing stage contributions.
• To investigate the impact of different brain regions of interest (ROIs) on reconstruction quality.

Main Methods:

• Implemented a gated recurrent unit (GRU) architecture for non-linear fMRI to VAE latent space mapping.
• Optimized the VAE latent space dimensionality.
• Conducted systematic evaluations of the first reconstruction stage's contribution and analyzed ROI-specific performance.
• Utilized the Natural Scenes Dataset (NSD) with fMRI data from eight subjects.

Main Results:

• The proposed architecture achieved competitive performance with an 85% reduction in the first stage's complexity.
• Sensitivity analysis confirmed the first stage's crucial role in maintaining structural similarity.
• Excluding early visual areas and focusing on semantic ROIs preserved semantics but reduced visual coherence.
• High inter-subject repeatability was observed (92% for visual, 98% for semantic metrics).

Conclusions:

• The study presents a key advancement in optimized neural decoding architectures for stimulus prediction using non-linear models.
• Analysis highlights the critical interplay between the two reconstruction stages.
• ROI-based analysis supports the two-stage AI model's reflection of the brain's hierarchical visual processing.