DeepMNF: Deep Multimodal Neuroimaging Framework for Diagnosing Autism Spectrum Disorder

The growing prevalence of neurological disorders, e.g., Autism Spectrum Disorder (ASD), demands robust computer-aided diagnosis (CAD) due to the diverse symptoms which require early intervention, particularly in young children. The absence of a benchmark neuroimaging diagnostics paves the way to study transitions in the brain's anatomical structure and neurological patterns associated with ASD. The existing CADs take advantage of the large-scale baseline dataset from the Autism Brain Imaging Data Exchange (ABIDE) repository to improve diagnostic performance, but the involvement of multisite data also amplifies the variabilities and heterogeneities that hinder satisfactory results. To resolve this problem, we propose a Deep Multimodal Neuroimaging Framework (DeepMNF) that employs Functional Magnetic Resonance Imaging (fMRI) and Structural Magnetic Resonance Imaging (sMRI) to integrate cross-modality spatiotemporal information by exploiting 2-dimensional time-series data along with 3-dimensional images. The purpose is to fuse complementary information that increases group differences and homogeneities. To the best of our knowledge, our DeepMNF achieves superior validation performance than the best reported result on the ABIDE-1 repository involving datasets from all available screening sites. In this work, we also demonstrate the performance of the studied modalities in a single model as well as their possible combinations to develop the multimodal framework.