SpeechCARE: dynamic multimodal modeling for cognitive screening in diverse linguistic and speech task contexts

Area of Science:

• Computational linguistics
• Artificial intelligence in healthcare
• Neuroscience

Background:

• Cognitive impairment, including Alzheimer's Disease and Related Dementias (ADRD) and Mild Cognitive Impairment (MCI), poses a significant global health challenge.
• Early detection is crucial for effective management and intervention.
• Speech analysis offers a non-invasive method for assessing cognitive function.

Purpose of the Study:

• To develop and evaluate SpeechCARE, a multimodal transformer pipeline for detecting cognitive impairment from speech recordings.
• To classify individuals into categories of ADRD, MCI, and healthy controls.
• To enhance the accuracy and scalability of cognitive impairment detection.

Main Methods:

• Utilized a multimodal transformer pipeline (SpeechCARE) integrating acoustic (mHuBERT) and linguistic (mGTE) embeddings with demographic data.
• Incorporated an advanced preprocessing pipeline including LLM-based audio anomaly detection, speech-task identification, noise reduction, and transcription.
• Employed a novel Adaptive Gating Fusion mechanism and a specialized encoding component for temporal pattern analysis.

Main Results:

• SpeechCARE achieved an average F1-score of 72.11% on a held-out test set from the NIA PREPARE challenge dataset.
• The model demonstrated strong multilingual generalizability across English, Spanish, and Mandarin.
• Threshold optimization improved Mild Cognitive Impairment (MCI) recall, and the model received special recognition from NIA.

Conclusions:

• SpeechCARE effectively detects cognitive impairment from speech, complementing existing biomarkers.
• The pipeline shows potential for early, scalable, and multilingual detection of cognitive decline.
• Further research and fairness analysis are warranted, particularly for diverse linguistic groups.