Artificial Intelligence Chatbots in Peritoneal Dialysis Education: A Cross-Sectional Comparative Study of Quality,...

Area of Science:

• Nephrology
• Artificial Intelligence
• Health Informatics

Background:

• Peritoneal dialysis (PD) is underutilized globally due to patient education gaps and misinformation.
• AI chatbots offer potential for enhancing health literacy and patient engagement in PD.
• Systematic evaluation of AI-generated PD content quality is lacking.

Purpose of the Study:

• To systematically assess the quality, reliability, and readability of AI-generated educational content on peritoneal dialysis.
• To compare the performance of multiple large language model-based chatbots in responding to PD-related queries.

Main Methods:

• Developed 45 expert-curated questions on PD across three domains.
• Utilized Gemini Pro 2.5, ChatGPT-5, and LLaMA Maverick 4 to generate responses.
• Evaluated responses using Flesch Reading Ease Score (FRES), Flesch-Kincaid Grade Level (FKGL), Ensuring Quality Information for Patients (EQIP), and Modified DISCERN instruments.

Main Results:

• Significant differences in readability and content quality were found among chatbots.
• Gemini Pro 2.5 demonstrated superior FRES, EQIP, and Modified DISCERN scores compared to ChatGPT-5 and LLaMA Maverick 4.
• LLaMA Maverick 4 exhibited the lowest performance across all evaluated metrics.

Conclusions:

• AI chatbots differ in their ability to provide readable and reliable information on peritoneal dialysis.
• AI-generated content can supplement PD patient education but requires careful clinical oversight.
• Further research is needed for safe AI integration into PD patient education, focusing on multilingual and outcome-based studies.