Fusing multimodal neuroimaging data with a variational autoencoder.

Area of Science:

• Neuroimaging
• Machine Learning
• Psychiatry

Background:

• Multimodal neuroimaging data offer comprehensive insights into brain function and structure.
• Integrating diverse data types presents challenges in dimensionality and information fusion.
• Understanding the interplay between neuroimaging modalities is crucial for diagnosing mental disorders.

Purpose of the Study:

• To develop a scalable and interpretable variational autoencoder (VAE) for fusing multimodal neuroimaging data.
• To reduce the dimensionality of fused data while preserving essential information.
• To evaluate the efficacy of the VAE-based dimensionality reduction for a schizophrenia classification task.

Main Methods:

• Utilized a variational autoencoder (VAE) to learn a latent representation from multimodal neuroimaging datasets.
• Included functional brain networks and structural magnetic resonance imaging (sMRI) data.
• Assessed the retained information by training a linear classifier on the reduced-dimension representations for schizophrenia classification.

Main Results:

• The VAE-based dimensionality reduction successfully retained meaningful information for classification.
• Achieved a high classification performance with an Area Under the Curve (AUC) of 0.8609 for schizophrenia detection.
• The proposed fusion method outperformed early and late fusion approaches using principal component analysis (PCA).

Conclusions:

• The VAE provides an effective and interpretable method for fusing multimodal neuroimaging data.
• Dimensionality reduction using VAEs can preserve critical information for clinical applications like mental disorder classification.
• This approach enables more rigorous analysis of complex multimodal neuroimaging data in clinical settings.