Multimodal Fusion and Transfer Learning for the Detection of Degenerative Parkinsonisms with Dopamine Transporter...

Dopamine transporter (DAT) SPECT is a validated biomarker for Parkinson's disease (PD) and related degenerative parkinsonisms. Interpretation relies on visual assessment supported by striatal image features such as striatal binding ratios (SBRs). Deep learning can aid this process but often underuses complementary data, lacks robustness to heterogeneous inputs, and offers limited interpretability. We built an end-to-end multimodal framework that encodes DAT images and scalar data (patient age and striatal image features) using, respectively, a vision transformer and a multilayer perceptron. A transformer-based fusion module then combined the encoded representations, while tackling possible missing inputs. Interpretability was provided through modality-level attention, spatial attention maps, occlusion analysis, and scalar feature saliency. Performance was evaluated on 664 Parkinson's Progression Markers Initiative (PPMI) cases and two local datasets A (N = 228) and B (N = 530) from different devices, including PD, atypical parkinsonisms, and non-degenerative subjects. Transfer learning involved pretraining on two datasets and finetuning on the third. On PPMI, the model reached 97.4% AUROC, 95.5% accuracy, 97.0% sensitivity, and 91.9% specificity, matching state-of-the-art performance. Results were similar on dataset B (98.6% AUROC) but lower on dataset A (92.6% AUROC), likely due to its smaller size and reduced image quality. Explainability analyses showed the model focused on clinically relevant striatal regions and identified key scalar features such as putamen SBR and asymmetry. The fusion module also supported stable predictions despite missing data. Our method efficiently combined multimodal data with heterogeneous datasets and partial multimodal data. Integrated explainability tools showed clinically meaningful attention that is expected to favor its adoption.