Multilingual Prediction of Cognitive Impairment with Large Language Models and Speech Analysis

Area of Science:

• Speech processing
• Artificial intelligence
• Neuroscience

Background:

• Cognitive impairment is a major global health concern requiring early detection.
• Current diagnostic methods are often subjective, expensive, and inaccessible.
• Previous speech analysis research has largely ignored multilingual contexts.

Purpose of the Study:

• To automatically detect mild cognitive impairment (MCI) and predict cognitive scores.
• To evaluate performance in both English and Chinese speakers.
• To explore the utility of speech foundation models for cognitive assessment.

Main Methods:

• Utilized Whisper, a speech foundation model, for language-agnostic speech embedding extraction.
• Employed ensemble models to integrate task-specific information.
• Participated in the INTERSPEECH 2024 TAUKADIAL Challenge with 169 participants.

Main Results:

• Achieved 81.83% unweighted average recall for MCI classification (2nd place).
• Obtained a root mean squared error of 1.196 for cognitive score prediction (1st place).
• Highlighted the importance of language-specific nuances over purely language-agnostic models.

Conclusions:

• Demonstrated the efficacy of language-specific ensemble modeling with Whisper embeddings.
• Showcased a scalable, non-invasive approach for cognitive health assessment.
• Achieved state-of-the-art results in multilingual cognitive impairment detection, relevant for Alzheimer's disease.