Retrieval-Augmented Multimodal Depression Detection.

Area of Science:

• Artificial Intelligence
• Computational Linguistics
• Psychology

Background:

• Multimodal deep learning shows promise for depression detection by integrating text, audio, and video.
• Existing methods using sentiment analysis face challenges like high computational costs and domain mismatch.
• Static knowledge limitations hinder the adaptability of current depression detection models.

Purpose of the Study:

• To propose a novel Retrieval-Augmented Generation (RAG) framework for depression detection.
• To enhance emotional representation and interpretability in multimodal depression detection.
• To overcome the limitations of existing sentiment analysis approaches in depression detection.

Main Methods:

• Developed a RAG framework integrating text, sentiment analysis, and Large Language Models (LLMs).
• Retrieved semantically relevant emotional content from a sentiment dataset.
• Generated an Emotion Prompt using an LLM as an auxiliary modality to enrich emotional representation.

Main Results:

• Achieved state-of-the-art performance on the AVEC 2019 dataset.
• Reported a Concordance Correlation Coefficient (CCC) of 0.593 and Mean Absolute Error (MAE) of 3.95.
• Outperformed previous transfer learning and multi-task learning baselines in depression detection.

Conclusions:

• The proposed RAG framework significantly improves depression detection accuracy and interpretability.
• Integrating LLM-generated Emotion Prompts offers a novel way to enhance multimodal analysis.
• This approach addresses key limitations of current sentiment analysis techniques in clinical applications.