A multimodal evidence-driven framework for clinical decision support in cognitive impairment

Area of Science:

• Artificial Intelligence in Medicine
• Neuroscience
• Medical Imaging Analysis

Background:

• Deep learning for cognitive impairment diagnosis shows promise but lacks interpretability and medical evidence linkage.
• Clinical translation of AI diagnostic tools is hindered by poor explainability and reliance on sparse data.

Purpose of the Study:

• To develop an interpretable AI framework for cognitive impairment diagnosis that integrates multimodal data and medical evidence.
• To enhance the accuracy and robustness of cognitive impairment diagnosis, especially in cases with incomplete or ambiguous information.

Main Methods:

• Developed the Multimodal Evidence-Driven Reasoning Framework (MEDRF), integrating a Multimodal Hierarchical Cascade (mHC) classifier with a retrieval-augmented large language model (RAG-LLM).
• Utilized routinely collected non-invasive data, including clinical profiles and structural MRI, for hierarchical diagnostic modeling.
• Evaluated mHC performance under feature masking and assessed RAG-LLM's corrective capabilities in data-sparse scenarios.

Main Results:

• The mHC classifier outperformed flat multimodal baselines across 15 diagnostic labels.
• RAG-LLM correction mitigated performance decline caused by feature masking, particularly under severe data sparsity.
• External validation showed improved accuracy from 0.706±0.038 to 0.753±0.032 by integrating RAG-LLM with mHC.
• The RAG module successfully resolved ambiguous predictions by retrieving analogous cases and textual evidence for multi-hop reasoning.

Conclusions:

• MEDRF offers a robust, interpretable framework for AI-assisted cognitive impairment diagnosis, synthesizing hierarchical prediction with evidence-grounded reasoning.
• The framework enhances decision support, proving particularly valuable for diagnostically ambiguous or incomplete clinical presentations.
• Physician reviews confirmed the favorable quality and perceived usefulness of MEDRF-generated reports for clinical practice.