Multi input-Multi output 3D CNN for dementia severity assessment with incomplete multimodal data.

Area of Science:

• Neuroimaging
• Artificial Intelligence in Medicine
• Neurology

Background:

• Alzheimer's Disease (AD) is the leading cause of dementia, progressing through distinct stages.
• Magnetic Resonance Imaging (MRI) and Positron Emission Tomography (PET) are crucial for early neurodegenerative disorder diagnosis, providing volumetric and metabolic brain information.
• Deep Learning (DL) shows promise in medical imaging, particularly Convolutional Neural Networks (CNNs), leading to Multimodal Deep Learning (MDL) for integrating diverse data sources.

Purpose of the Study:

• To systematically analyze Multimodal Deep Learning (MDL) approaches for dementia severity assessment using MRI and PET scans.
• To propose and evaluate a novel 3D CNN architecture capable of handling incomplete multimodal data.

Main Methods:

• A Multi Input-Multi Output 3D CNN was developed for dementia severity assessment.
• The network's training adapts to input characteristics, enabling it to process incomplete MRI or PET acquisitions.
• Experiments were conducted on the OASIS-3 dataset to validate the proposed MDL approach.

Main Results:

• The proposed 3D CNN demonstrated satisfactory performance in dementia severity assessment.
• The MDL approach outperformed methods relying on single imaging modalities (MRI or PET alone).
• The network effectively handled cases with missing one imaging modality, showcasing its robustness.

Conclusions:

• Multimodal Deep Learning offers a powerful framework for dementia severity assessment.
• The developed 3D CNN effectively integrates MRI and PET data, even with missing modalities.
• This approach shows significant potential for improving the accuracy and flexibility of diagnosing neurodegenerative disorders.