Adjacent-Aware Modality Recovery Based on Incomplete Multi-Modal Brain Disease Diagnosis

Area of Science:

• Neuroscience
• Medical Imaging
• Machine Learning

Background:

• Multi-modal learning aids in diagnosing brain diseases like epilepsy and Alzheimer's.
• Incomplete data, where some modalities are missing, hinders conventional diagnostic methods.
• Existing methods often ignore semantic relationships and latent information in missing data.

Purpose of the Study:

• To propose an adjacent-aware distillation recovery framework for incomplete multi-modal learning.
• To enhance the diagnosis of brain diseases, specifically epilepsy and Alzheimer's disease, despite data limitations.
• To address the limitations of conventional methods in handling incomplete multi-modal data.

Main Methods:

• Developed a novel framework integrating adjacent-aware modality recovery and multi-modal representation learning.
• Introduced a label-guided adjacent-aware recovery module utilizing self-attention for neighbor semantics.
• Employed knowledge distillation to refine recovered features and enhance generalization under data incompleteness.

Main Results:

• The proposed framework effectively reconstructs missing modalities and improves diagnostic performance.
• Demonstrated significant effectiveness in diagnosing epilepsy and Alzheimer's disease with incomplete data.
• The joint pipeline enhances feature extraction and representation by fusing original and recovered modality information.

Conclusions:

• The adjacent-aware distillation recovery framework offers a robust solution for incomplete multi-modal learning in brain disease diagnosis.
• The method shows promise for improving diagnostic accuracy in real-world scenarios with missing data.
• This approach advances the field of multi-modal learning for neurological disorder diagnosis.