Functional Alignment-Auxiliary Generative Adversarial Network-Based Visual Stimuli Reconstruction via Multi-Subject...

Area of Science:

• Neuroscience
• Computer Science
• Machine Learning

Background:

• Functional Magnetic Resonance Imaging (fMRI) offers high spatial and temporal resolution for brain activity measurement.
• fMRI data exhibits significant inter-subject heterogeneity, challenging multi-subject analysis.
• Existing methods often overlook this heterogeneity, leading to suboptimal decoding results.

Purpose of the Study:

• To propose a novel multi-subject approach for visual image reconstruction using fMRI data.
• To address and alleviate inter-subject heterogeneity in fMRI responses.
• To enhance the reliability and applicability of multi-subject decoding.

Main Methods:

• Developed a functional alignment-auxiliary generative adversarial network (FAA-GAN).
• Incorporated a generative adversarial network (GAN) module for visual stimulus reconstruction.
• Implemented a multi-subject functional alignment module to standardize fMRI response spaces.
• Utilized a cross-modal hashing retrieval module for data similarity analysis.

Main Results:

• The FAA-GAN method demonstrated superior performance compared to existing deep learning-based reconstruction techniques.
• Functional alignment effectively reduced heterogeneity among subjects in fMRI data.
• The approach successfully reconstructed visual stimuli from brain activity.

Conclusions:

• The proposed FAA-GAN method offers a robust solution for multi-subject visual image reconstruction from fMRI.
• Addressing subject heterogeneity is crucial for improving the accuracy of brain decoding.
• FAA-GAN advances the field of neuroimaging analysis and brain-computer interfaces.