A Fusion of Multi-view 2D and 3D Convolution Neural Network based MRI for Alzheimer's Disease Diagnosis

Alzheimer's disease (AD) is a neurodegenerative disease leading to irreversible and progressive brain damage. Close monitoring is essential for slowing down the progression of AD. Magnetic Resonance Imaging (MRI) has been widely used for AD diagnosis and disease monitoring. Previous studies usually focused on extracting features from whole image or specific slices separately, but ignore the characteristics of each slice from multiple perspectives and the complementarity between features at different scales. In this study, we proposed a novel classification method based on the fusion of multi-view 2D and 3D convolutions for MRI-based AD diagnosis. Specifically, we first use multiple sub-networks to extract the local slice-level feature of each slice in different dimensions. Then a 3D convolution network was used to extract the global subject-level information of MRI. Finally, local and global information were fused to acquire more discriminative features. Experiments conducted on the ADNI-1 and ADNI-2 dataset demonstrated the superiority of this proposed model over other state-of-the-art methods for their ability to discriminate AD and Normal Controls (NC). Our model achieves 90.2% and 85.2% of accuracy on ADNI-2 and ADNI-1 respectively, thus it can be effective in AD diagnosis. The source code of our model is freely available at https://github.com/fengduqianhe/ADMultiView.