A Comparison of a Customized Peripheral Artery Disease (PAD)-Specific Generative AI Chatbot and General-Purpose AI...

Background/Objective: Patients with peripheral artery disease (PAD) are known to have poor awareness and understanding of the diagnosis. The role of generative AI chatbots in improving PAD patient education is unknown. Our goal is to compare a generative AI chatbot customized for PAD patient education to publicly available AI chatbots. Methods: This is a cross-sectional comparative evaluation of the responses of four AI chatbots to ten prompts that are commonly asked questions about PAD. The three publicly available AI chatbots were ChatGPT-5, Gemini 2.5 Flash, and Claude Sonnet 4.5. We created a customized, voice AI chatbot for PAD education grounded on curated and prompt-injected guidance called Vascular Education and Resources using Artificial Intelligence, or "VERA." De-identified chatbot-generated responses to inputs were assessed for readability (Flesch-Kincaid Grade Level, Flesch Reading Ease, Gunning Fog Index, Simple Measure of Gobbledygook Index, and Average Reading Level Consensus Score), accuracy, comprehensiveness, and patient education quality (Patient Education Materials Assessment Tool; PEMAT) using validated instruments and expert scoring rubrics. Nonparametric statistical testing was used to compare chatbot performance across all evaluation domains. Results: VERA generated the most accessible text compared to the other chatbots and produced responses at a median grade level of 6.6, which was lower than responses from the other chatbots. PAD expert-rated accuracy scores were high across all the chatbots without significant differences between them. Comprehensiveness scores were more varied and demonstrated that VERA was less comprehensive than the other chatbots. PEMAT understandability scores were uniformly high. PEMAT actionability scores were low overall but did not differ significantly across chatbots on post hoc analysis. Conclusions: A generative AI chatbot research tool customized for PAD patient education generates textual information about PAD that is more accessible (mean grade level 6.6) than publicly available AI chatbots without loss of accuracy, albeit with modestly reduced comprehensiveness that reflects intentional simplification for patient-centered communication. Future research will assess the acceptability and feasibility of this research tool to be adopted as part of PAD patient education.