Evaluation of a Clinical Protocol-Guided Chatbot to Support Telecarers in Diabetes Management: Embedded Mixed Methods...

Type 2 diabetes mellitus (T2DM) is a chronic condition that requires ongoing self-management. While chatbots offer scalable monitoring and management, evidence on their integration into structured hospital tele-support workflows, where patient engagement and digital readiness may differ from those in standalone digital interventions, remains limited.

This study aimed to describe the design of a clinical protocol-guided chatbot, Sharing Information with Users on Diabetes Actions and Interventions (Siu Dai), and evaluate its feasibility, usability, and acceptability among patients and telecarers within a hospital tele-support program.

We used an embedded mixed methods design (QUAL [quan]) guided by the World Health Organization (WHO) monitoring and evaluating digital health interventions and the theoretical framework of acceptability. Through a co-design process involving clinicians, researchers, telecarers, and developers, the chatbot was aligned with established clinical protocols. It featured 7 precall sessions designed to collect patient data, generate summary reports for staff, and deliver targeted educational content. Patients (≥21-year-old individuals with T2DM) and telecarer nurses (with experience ≥6 mo) were recruited from an existing tele-support program. Data sources included interviews, online surveys (5-point Likert scale), and chatbot usage logs. Qualitative data were analyzed thematically; quantitative data were summarized descriptively to supplement and contextualize the qualitative findings.

Fourteen participants (7 patients and 7 telecarers) completed both interviews and surveys and were included in the analysis. Patients (mean age 45.3, SD 15.4 y) had a median diabetes duration of 10 (IQR 3-25; range 1-30) years, and all were digitally literate. Telecarers had a median of 5 (IQR 4-9) years of telehealth experience. Quantitatively, patients showed high engagement, completing 79% of started sessions with a median completion time of 2 minutes and 58 seconds. Patient satisfaction was high, with 86% (6/7) willing to recommend the tool; however, telecarer endorsement for universal rollout to patients in other clinical programs or outpatient clinics was more cautious (3/7, 43%). Qualitative findings revealed 5 themes: (1) perceptions of chatbot usability, (2) alignment and integration into telecarers' processes, (3) factors influencing acceptability of the chatbot, (4) perceived efficacy of the chatbot for T2DM self-management and care delivery, and (5) suggested improvements and enhancements. Integrated findings revealed that the chatbot was usable and functioned as a precall support tool that optimized the personalization of tele-support sessions. However, its perceived efficacy in self-management behavior was limited by this cohort's extensive experience with T2DM management. Key barriers to adoption included administrative burden for telecarers due to a lack of electronic medical record integration and content redundancy for patients. Suggested enhancements included multilingual support and adaptive content catering to different levels of disease-specific literacy.

A clinical protocol-guided chatbot is feasible for hospital tele-support, complementing rather than replacing human care. It automates routine assessments and reinforces education, but future iterations must prioritize workflow integration and adaptive content to sustain engagement.