Feasibility and user evaluation of HopeBot: An LLM-powered conversational chatbot for depression screening

Area of Science:

• Digital Health
• Artificial Intelligence in Healthcare
• Mental Health Screening

Background:

• Clinician-administered Patient Health Questionnaire-9 (PHQ-9) interviews are accurate but not scalable.
• Self-administered PHQ-9 lacks interaction and clarification, potentially affecting engagement and accuracy.
• There is a need for scalable, interactive depression screening tools.

Purpose of the Study:

• To develop and assess the feasibility and acceptability of HopeBot, a conversational chatbot for administering the PHQ-9.
• To evaluate the score concordance between chatbot-assisted and self-administered PHQ-9.
• To gauge user confidence and usability of the chatbot for depression screening.

Main Methods:

• Developed HopeBot, a large language model (LLM) chatbot with retrieval-augmented generation (RAG) for real-time clarification and safety.
• Conducted a within-subject feasibility study with 132 adults comparing chatbot-assisted and self-administered PHQ-9.
• Collected user feedback via an evaluation survey and a comparative feedback module.

Main Results:

• Chatbot and self-report PHQ-9 scores showed high concordance (ICC=0.92).
• 71% of participants reported greater confidence in chatbot-assisted scores due to clearer structure and supportive tone.
• Usability ratings were favorable, with high scores for comfort, voice clarity, and handling sensitive topics.

Conclusions:

• An LLM-powered conversational agent with RAG can feasibly and acceptably administer the PHQ-9, demonstrating strong score concordance.
• HopeBot shows potential as a scalable adjunct for low-burden, early depression screening.
• Further validation against clinician-administered assessments in real-world settings is recommended.