Unveil Multi-Picture Descriptions for Multilingual Mild Cognitive Impairment Detection via Contrastive Learning.

Area of Science:

• Neurology
• Artificial Intelligence
• Computational Linguistics

Background:

• Detecting Mild Cognitive Impairment (MCI) is crucial for early intervention.
• Previous MCI detection methods primarily focused on single-picture descriptions from English speakers.
• Multilingual and multi-picture settings present significant challenges for MCI detection due to complex content analysis.

Purpose of the Study:

• To develop and evaluate a novel framework for improved MCI detection in multilingual and multi-picture settings.
• To address the limitations of prior work by incorporating diverse linguistic and visual data.
• To enhance the accuracy and robustness of MCI detection systems.

Main Methods:

• Proposed a framework incorporating supervised contrastive learning for enhanced representation learning.
• Integrated image modality alongside speech and text modalities, moving beyond unimodal approaches.
• Applied a Product of Experts (PoE) strategy to reduce spurious correlations and prevent overfitting.

Main Results:

• Achieved a +7.1% increase in Unweighted Average Recall (UAR), reaching 75.2% from 68.1%.
• Improved F1 score by +2.9%, from 80.6% to 83.5%, compared to a text unimodal baseline.
• Demonstrated that contrastive learning provided greater performance gains for text modality than for speech modality.

Conclusions:

• The proposed framework significantly enhances MCI detection performance in challenging multilingual and multi-picture contexts.
• The integration of multiple modalities and advanced learning strategies is effective for improving diagnostic accuracy.
• This work provides a promising direction for developing more inclusive and accurate MCI detection tools.