Cross-subject fMRI-to-Image with Visual-cortex 2D Representation and Pre-Training.

Area of Science:

• Neuroscience
• Neuroimaging
• Machine Learning

Background:

• Decoding brain activity from functional Magnetic Resonance Imaging (fMRI) is vital for neuroscience.
• Traditional methods often analyze data from individual subjects, limiting cross-subject applicability.
• Existing approaches struggle with inter-subject variability in fMRI signals.

Purpose of the Study:

• To introduce X-MinD, a novel zero-shot, cross-subject fMRI-to-image framework.
• To reconstruct images from brain activity in unseen subjects.
• To advance the field of cross-subject brain decoding.

Main Methods:

• Converting fMRI visual cortex signals into 2D surface images for analysis.
• Utilizing structured visual textures for brain activity representation.
• Developing a customized visual autoencoder for self-supervised pre-training of fMRI features.
• Employing a linear mapping between fMRI and image feature spaces for decoding.

Main Results:

• X-MinD successfully reconstructs realistic and semantically consistent images from cross-subject fMRI data.
• The framework achieves effective zero-shot decoding without requiring model fine-tuning.
• Experimental results demonstrate superior performance in cross-subject brain decoding compared to existing methods.

Conclusions:

• The proposed fMRI representation mitigates inter-individual differences.
• X-MinD enables generalizable learning of visual cortex fMRI features.
• This approach offers a significant advancement in decoding brain activity across subjects.