Cogformer: A Unified Multi-Scale Brain Representation for Visual Decoding and Reconstruction From fMRI.

Area of Science:

• Neuroscience and Artificial Intelligence
• Computational Neuroscience
• Machine Learning for Brain Imaging

Background:

• Deep generative models (DGMs) have advanced fMRI decoding but face challenges like limited data and low signal-to-noise ratios.
• Accurate brain activity representation is crucial for understanding neural processes.
• Existing methods struggle with the complexity and noise inherent in fMRI data.

Purpose of the Study:

• To introduce Cogformer, a novel multi-scale brain representation method for enhanced fMRI decoding.
• To address limitations of current DGMs in handling fMRI data challenges.
• To improve the accuracy and generalization of reconstructing and decoding brain activity.

Main Methods:

• Developed Cogformer, utilizing self-attention for multi-scale fMRI activity representation.
• Integrated a synchronized decoding and dynamic decoupling strategy via cross-attention for structural and semantic features.
• Employed a prior diffusion module to enhance semantic alignment in challenging tasks like image captioning and reconstruction.

Main Results:

• Cogformer achieved superior performance in category classification, multi-label classification, and image retrieval compared to transformer baselines.
• Demonstrated competitive performance against state-of-the-art methods in image captioning and reconstruction.
• Showcased strong decoding capabilities and generalization potential across a wide range of visual decoding tasks.

Conclusions:

• Cogformer offers a powerful and unified approach to multi-scale brain representation from fMRI data.
• The method effectively tackles challenges of limited samples and low signal-to-noise ratios in fMRI.
• Cogformer represents a significant advancement in decoding brain activity for diverse visual tasks.