ROI-informed cross-modal supervised contrastive learning for multimodal depression classification using rs-fMRI and...

Neuroimaging-based classification of depression-related symptom and diagnostic groups remains challenging because of clinical and neurobiological heterogeneity. Structural MRI and resting-state functional MRI provide complementary information, but most multimodal approaches rely on global fusion and do not incorporate region-level (ROI) information into cross-modal alignment. We propose a two-stage multimodal framework for depression classification based on ROI-informed cross-modal supervised contrastive learning (RCSCL), which does not require explicit ROI-to-ROI correspondence. Utilizing the Schaefer 200 atlas, we extracted ROI tokens using a MedicalNet-pretrained 3D ResNet-18 for structural MRI and a graph attention network for resting-state fMRI. The tokens were aggregated with attention pooling and the resulting modality-level representations were aligned in a shared embedding space using class-conditional supervised contrastive learning. Global features were then fused using self-attention, and the classification model was optimized using focal loss to address class imbalance. The framework was evaluated on an in-house single-center dataset of 500 subjects and the publicly available SRPBS multi-site MDD dataset. In the primary attention-based implementation, RCSCL significantly increased AUC/AUPRC from 0.805/0.326 to 0.855/0.410 in the in-house dataset and from 0.736/0.704 to 0.766/0.744 on SRPBS, numerically exceeding those of all evaluated fusion baselines without RCSCL; extending RCSCL to these backbones produced further but backbone-dependent gains. Hierarchical visualization analyses identified default mode, limbic, and prefrontal regions as recurrently salient, consistent with prior depression neuroimaging findings, although these attention-derived importance values are model-internal indices. These results suggest that ROI-informed multimodal contrastive learning improves class-discriminative representation learning for multimodal depression classification.