On-premises open-source large language models for privacy-preserving multimodal depression screening.

Area of Science:

• Artificial Intelligence
• Clinical Psychology
• Computational Linguistics

Background:

• Depression screening is crucial, but privacy concerns limit clinical use of closed models.
• Existing large language model (LLM) approaches are text-centric, lacking multimodal integration.
• Multimodal integration of acoustic features and clinical data with LLMs is limited.

Purpose of the Study:

• Develop and externally validate a privacy-preserving multimodal depression screening framework.
• Utilize open-source LLMs integrating sociodemographic data, emotion-memory narratives, and acoustic features.
• Address limitations of text-centric and closed-model approaches in depression screening.

Main Methods:

• Analyzed 3536 participants, combining sociodemographic variables, Korean narrative transcripts, and acoustic features (eGeMAPS).
• Selected statistically significant features and evaluated five open-source LLMs (Gemma-3-27B, Qwen-3-32B, Llama-3.3-70B, Phi4-14B, gpt-oss-20b) using zero-shot, Chain-of-Thought, and supervised fine-tuning.
• Performed external validation on the Extended Distress Analysis Interview Corpus (E-DAIC) with 275 participants.

Main Results:

• Supervised fine-tuning improved all models, achieving internal accuracies of 0.852-0.881 and F1-scores of 0.818-0.865.
• F1-scores increased by 0.12-0.30 compared to prompting-only methods.
• External validation showed accuracies of 0.764-0.822 and F1-scores of 0.683-0.807.

Conclusions:

• Multimodal open-source LLMs integrating clinical data, narratives, and acoustics support privacy-preserving depression screening.
• Supervised fine-tuning consistently enhanced model performance.
• External validation confirmed the framework's robustness beyond the initial cohort.