Development and validation of AI-based chatbot for self-management in liver transplant recipients

Liver transplant recipients require complex self-management after transplantation. This study aimed to validate an artificial intelligence (AI)-based chatbot designed to support self-management.

A mixed-methods study was conducted at a medical center in Northern Taiwan (October 2022-December 2024). The chatbot was developed through four stages: creation of a self-management Q&A database, system development, validation, and usability testing. Interview findings from patients and healthcare professionals informed the database. The chatbot was deployed via LINE@ using a hybrid framework that retrieves validated responses from a predefined database and uses ChatGPT-4o-mini only for unmatched queries. Content validity index (CVI), intraclass correlation coefficient (ICC), Cohen's kappa, satisfaction, usability, and intention to use were evaluated.

A total of 112 questions were classified into four domains and 21 subcategories. Two rounds of expert validation demonstrated acceptable-to-excellent agreement. The CVI improved from 0.79 to 0.88, ICC increased from 0.957 to 0.976, and Cohen's kappa reached 0.88. The satisfaction score was 4.71 (standard deviation [SD] = 0.39) on a 5-point Likert scale. The usability and intention-to-use scores were 6.04 (SD = 0.98) and 6.12 (SD = 1.02), respectively, on a 7-point scale.

The AI-based chatbot demonstrated strong validity and usability, supporting its readiness for larger-scale clinical evaluation. This hybrid approach shows promise for improving post-discharge self-management and personalized support for liver transplant recipients.