Leveraging large language models for automated depression screening.

Area of Science:

• Artificial Intelligence in Mental Health
• Natural Language Processing Applications
• Computational Psychiatry

Background:

• Mental health diagnoses present unique management challenges impacting well-being and daily functioning.
• Self-report questionnaires are standard for mental health screening but rely on subjective, potentially biased responses.
• Quantifying self-reported experiences via natural language processing has faced accuracy limitations despite LLM advancements.

Purpose of the Study:

• To evaluate the effectiveness of zero-shot learning Large Language Models (LLMs) for screening and assessing depression using item scales.
• To demonstrate the potential of LLMs in predicting self-reported questionnaire scores from clinical interview data.

Main Methods:

• Utilized the DAIC-WOZ dataset, a public resource with clinical interview transcripts and self-report questionnaire data.
• Employed the RISEN prompt engineering framework to assess LLMs' predictive capabilities for depression symptoms (PHQ-8 items).
• Evaluated multiple LLMs including GPT models, Llama3_8B, Cohere, and Gemini based on accuracy and F1 scores.

Main Results:

• GPT models, particularly GPT-4o, outperformed other LLMs (Llama3_8B, Cohere, Gemini) with an average accuracy of 75.9% and an F1 score of 0.74 across all PHQ-8 items.
• GPT models effectively predicted emotional and cognitive symptom items.
• Llama 3_8B excelled at detecting anhedonia symptoms, while Cohere showed strength in identifying psychomotor activity symptoms.

Conclusions:

• LLMs demonstrate significant potential for assisting in depression screening by predicting self-reported questionnaire scores from textual interview data.
• Preliminary performance suggests LLMs can be valuable tools in mental healthcare, though further research is needed.
• Future work should focus on developing frameworks for specific LLM applications to mental health symptoms and exploring additional datasets with model fine-tuning.