The effectiveness of a sentence completion test for depression screening using large language models.

Area of Science:

• Mental Health
• Artificial Intelligence
• Psychometrics

Background:

• Depressive symptoms are a global mental health concern, significantly impacting Thailand.
• Traditional depression screening methods can be subjective and prone to bias.
• Large language models (LLMs) offer potential for modernizing mental health assessments.

Purpose of the Study:

• To evaluate the efficacy of a novel sentence completion test for depression detection using LLMs.
• To enhance objectivity and reduce bias in depression screening.
• To identify key life areas (family, society, health, self-concept) associated with depression risk.

Main Methods:

• A new depression sentence completion test was developed and administered to 373 participants (aged 20-40).
• Four LLMs (LLAMA 3.1-8B, Gemma2-9B, Qwen2-7B, Typhoon1.5-7B) were employed for analysis.
• Random forest and decision tree classifiers were used to classify depression risk.

Main Results:

• Health (0.48), self-concept (0.49), and DIFF (0.54) showed strong positive correlations with sentiment levels, indicating significant depression risk indicators.
• Family (0.27) and society (0.19) also showed positive correlations, though weaker.
• Models demonstrated reliability with a 0.78 lower bound accuracy (p ≤ .05); LLAMA3.1 and Gemma2 exhibited the highest sensitivity.

Conclusions:

• LLMs show promise in objectively assessing depression risk through sentence completion tests.
• Health and self-concept are critical factors in identifying depression risk.
• Future research should focus on ethical considerations, diverse populations, and data updates for improved accuracy and generalizability.