Explaining Multimodal Features for Screening of Cognitive Impairment Using Shapley Values.

Area of Science:

• Neuroimaging
• Speech analysis
• Machine learning for healthcare

Background:

• Current Alzheimer's disease (AD) screening often relies on single data types like medical imaging or speech.
• Previous research indicates that combining multiple data sources (multimodal approaches) yields superior performance compared to single-modality models.
• Understanding the contribution of each modality is crucial for developing robust and interpretable AI models for AD screening.

Purpose of the Study:

• To investigate feature importance in multimodal AI models for Alzheimer's disease screening using explainable AI (XAI) techniques.
• To determine the influence of brain imaging versus speech features on model decisions at both input and fusion levels for state and predictive screening.
• To enhance the reliability and clinical interpretability of multimodal AI models for AD detection.

Main Methods:

• Utilized explainable AI (XAI) techniques, specifically Shapley values (SHAP, GradSHAP, DeepSHAP), to analyze feature importance.
• Developed and evaluated multimodal machine learning models integrating structural brain imaging and conversational speech data.
• Applied both local (individual patient) and global (aggregated) analyses to identify key predictive features.

Main Results:

• Multimodal models integrating brain imaging and speech data demonstrated superior performance in Alzheimer's disease screening.
• Volumetric features from brain imaging were found to be more influential in classification than acoustic and linguistic features from speech.
• XAI analysis provided insights into feature contributions at both the input and fusion levels for state and predictive screening.

Conclusions:

• Combining structural brain imaging and speech data significantly improves Alzheimer's disease screening accuracy.
• Explainable AI methods are vital for understanding and validating multimodal AI models in clinical settings.
• Brain imaging features play a more critical role than speech features in the current multimodal screening models, guiding future research and clinical application.