Related Experiment Video
Updated: Oct 21, 2026
Evidence-based Knowledge Synthesis and Hypothesis Validation: Navigating Biomedical Knowledge Bases via Explainable AI and Agentic Systems
Published on: June 13, 2025
ChatGPT Versus MediSearch: A Comparative Analysis of Artificial Intelligence Platforms for Patient Education in
Akshar P Patel1, Ishan Aggarwal1, Stanley Joseph1
1Department of Anesthesiology and Perioperative Medicine, Medical College of Georgia at Augusta University, Augusta, GA.
Objectives:
To evaluate the reliability and readability of responses generated by two artificial intelligence (AI) platforms-ChatGPT and MediSearch-to common patient-focused questions about cardiac surgery and its anesthetic implications.
Design:
Prospective, comparative, cross-sectional evaluation.
Setting:
Single-institution academic study; no direct patient involvement. Questions were sourced from publicly available online data (Google "People Also Ask" feature).
Participants:
One hundred unique patient-focused questions about cardiac surgery and anesthesia, categorized using the Rothwell classification system (80 fact, 14 policy, 6 value).
Interventions:
Each question was submitted individually to the free versions of ChatGPT (version 4.0) and MediSearch. Responses were evaluated using three validated metrics: reliability, assessed using the modified DISCERN tool; readability, assessed using the Flesch-Kincaid grade level (FK) and Coleman-Liau index (CL); and clinical accuracy, assessed by a board-certified cardiothoracic anesthesiologist and a board-certified cardiothoracic surgeon using a 3-point Likert scale.
Measurements And Main Results:
MediSearch demonstrated significantly higher reliability (mean DISCERN score, 4.64 v 2.47; p < 0.001). ChatGPT generated significantly more readable responses (mean FK score, 10.33 v 11.34 [p < 0.01]; mean CL score, 15.72 v 17.30 [p < 0.001]) and significantly higher clinical accuracy (mean, 0.82 v 0.71; p < 0.01). Subgroup analysis by Rothwell category confirmed MediSearch's reliability advantage across all question types, while ChatGPT's readability advantage was most pronounced for fact-based questions.
Conclusions:
This study reveals a critical trade-off between evidence-based reliability and linguistic accessibility in AI-generated cardiac surgery patient education content. MediSearch offers superior reliability, while ChatGPT provides more readable and accurate responses. Neither platform achieves the sixth-grade readability level recommended for broad public accessibility. These findings underscore the need for AI platforms to balance clinical accuracy with health literacy standards before being endorsed as patient education tools in high-stakes settings.