Competency of Large Language Models in Evaluating Appropriate Responses to Suicidal Ideation: Comparative Study

Area of Science:

• Artificial Intelligence in Mental Health
• Natural Language Processing Applications
• Clinical Psychology Research

Background:

• Rising suicide rates in the US prompt individuals to seek support from large language models (LLMs).
• There is a growing need to understand the capabilities of LLMs in sensitive mental health contexts.

Purpose of the Study:

• To evaluate the accuracy of three major LLMs in distinguishing appropriate from inappropriate responses to suicidal ideation.
• To compare LLM performance against expert suicidologist ratings.

Main Methods:

• An observational, cross-sectional study utilized the revised Suicidal Ideation Response Inventory (SIRI-2) with 24 scenarios.
• ChatGPT-4o, Claude 3.5 Sonnet, and Gemini 1.5 Pro rated clinician responses to hypothetical patient situations.
• LLM ratings were compared to expert suicidologist scores using linear regression and z-score analysis.

Main Results:

• All three LLMs exhibited an upward bias, rating responses as more appropriate than expert suicidologists did.
• Outlier analysis revealed 19% of ChatGPT, 11% of Claude, and 36% of Gemini responses differed significantly from expert ratings.
• Claude and ChatGPT achieved scores comparable to or exceeding mental health professionals, while Gemini scored similarly to untrained staff.

Conclusions:

• Current LLMs demonstrate a tendency to overestimate response appropriateness in suicidal ideation contexts.
• Despite bias, Claude 3.5 Sonnet and ChatGPT-4o performance aligns with or surpasses that of trained mental health professionals.
• Further research is needed to refine LLM capabilities for safe and effective mental health support.