Exploring the potential of ChatGPT as a digital advisor in acute psychiatric crises: a feasibility study

This exploratory study tested ChatGPT as a digital advisor chatbot for German-speaking individuals in acute psychiatric crises. Additionally, the attitudes of young physicians and psychologists towards the use of large language models (LLMs) in healthcare were investigated.

In total, 20 resident physicians and psychologists simulated patients in three clinical scenarios (depression, psychosis, adjustment disorder) and interacted with ChatGPT. They evaluated the chatbot's performance regarding overall experience, pleasantness, appropriateness of the responses, realism, and helpfulness. Before and after the intervention, their attitudes towards such a chatbot were assessed. Finally, they assessed 12 statements about the future of LLMs in healthcare and provided open feedback on the chat experience.

ChatGPT received predominantly positive ratings (over 8/10 points) for overall experience, helpfulness, pleasantness, and appropriateness, while realism was rated slightly lower at 7/10 points. The appropriateness of the responses varied significantly between the scenarios, with lower ratings for the psychosis scenario. Open feedback confirmed the limited suitability of ChatGPT for psychosis patients. Overall, 70% or more of the participants agreed that LLMs will become increasingly important in everyday life and healthcare, and that an LLM-based chatbot would be a modern tool for low-threshold access to initial psychiatric aid. However, the high number of neutral responses across all 12 items (20-45%) indicates uncertainty regarding the actual benefits and risks.

The performance of ChatGPT was rated positively overall by the participants. Significant practical and methodological limitations remain, however, highlighting the need for further research including real patients for a gradual, carefully monitored integration of LLMs into mental healthcare.