X-ViTCNN: A Novel Network-Level Fusion of Transfer Learning and Customized Vision Transformer for Multi-Stage...

Background/Objectives: Alzheimer's disease (AD), the most prevalent form of dementia, is characterized by an overall decline in cognitive functioning and represents a major public health crisis. It remains critical to be able to accurately and quickly diagnose patients with AD; however, recent deep learning approaches using MRI data do not provide sample generalization, have high computational requirements, and offer little interpretability. Methods: In this study, we present a new framework called eXplorative ViT-CNN (X-ViTCNN) that combines a customized Vision Transformer model with two previously trained CNNs (DenseNet201 and MobileNetV2). With our proposed preprocessing approach using contrast-enhanced preprocessing to highlight neuroanatomical features as well as Bayesian Optimization to tune hyperparameters, we fuse local structural features originating from the CNNs with global representations from the transformer and feed the final result to fully connected dense layers for multi-stage classification. We also use Grad-CAM visualizations to provide insight into how our model arrived at its classification. Results: Experiments conducted on ADNI and OASIS datasets demonstrate the superiority of X-ViTCNN, achieving accuracies of 97.98% and 94.52%, respectively. The model outperformed individual baselines and other pre-trained architectures, showing balanced sensitivity and specificity across all AD stages. Conclusions: The proposed X-ViTCNN framework is a powerful, interpretable method for predicting the development of multi-stage Alzheimer's disease using MRI scans. The combination of complementary feature learning, automatic hyperparameter optimization and interpretability through visualization make it an excellent potential tool for clinicians to support their decision making in the early diagnosis and ongoing monitoring of persons with Alzheimer's disease.