Contrastive and Transfer Learning for Aligned Multimodal Neuroimaging Classification of Autism Spectrum Disorder

Area of Science:

• Neuroscience
• Medical Imaging
• Machine Learning

Background:

• Autism Spectrum Disorder (ASD) assessment is challenging due to observer-dependent behavioral tools and heterogeneous neuroimaging data.
• Existing methods struggle with integrating multimodal neuroimaging data for accurate ASD classification.

Purpose of the Study:

• To introduce Fuse After Aligned (FAA), a novel multimodal classification framework for objective ASD diagnosis.
• To enhance ASD classification by aligning structural MRI (sMRI) and functional connectivity (FC) features before fusion using a contrastive objective.

Main Methods:

• Developed FAA, a framework combining transfer learning for sMRI representation and contrastive learning for pre-fusion alignment of sMRI and resting-state fMRI-derived FC features.
• Evaluated FAA on the ABIDE-I NYU subset (75 ASD, 98 controls) using a five-fold cross-validation protocol.
• Compared FAA performance against naive fusion and sMRI-only baselines, and assessed the impact of the contrastive objective and different deep learning architectures (ResNet-18, ViT-16).

Main Results:

• FAA achieved a mean accuracy of 92.6%, outperforming naive fusion (87.4%) and sMRI-only baseline (90.9%).
• Ablation studies confirmed that the contrastive objective significantly improved classification performance.
• ResNet-18 demonstrated superior performance over ViT-16 in this small-sample neuroimaging setting.

Conclusions:

• Pre-fusion feature alignment using a contrastive objective enhances multimodal classification for ASD.
• The FAA framework provides a robust, computationally efficient, and clinically viable method for objective ASD diagnosis.
• FAA shows strong potential for generalization to multi-site neuroimaging studies, advancing ASD assessment.