AANet: Attentive All-level Fusion Deep Neural Network Approach for Multi-modality Early Alzheimer's Disease Diagnosis

Multi-modality deep learning models have recently been used for disease diagnosis; however, effectively integrating diverse, complex, and heterogeneous data remains a challenge. In this study, we propose a novel system, attentive All-level Fusion(AANet), to fuse multi-level and multi-modality patient data, including 3D brain images, patient demographics, genetics, and blood biomarkers into a deep-learning framework for disease diagnosis, and tested it for early Alzheimer's disease diagnosis. We first constructed a deep learning feature pyramid network for whole-brain brain magnetic resonance imaging (MRI) feature extraction. We then leveraged the self-attention-based all-level fusion method by automatically adjusting weights of all-level MRI image features, patient demographics, blood biomarkers, and genetic data. We trained and tested AANet on data from the Alzheimer's Disease Neuroimaging Initiative for the task of classifying mild cognitive impairment from Alzheimer's disease, a challenging task in early Alzheimer's disease diagnosis. AANet achieved an accuracy of 90.5%, outperformed several state-of-the-art methods. In summary, AANet provides an advanced methodological framework for multi-modality-based disease diagnosis.