Related Experiment Video
Updated: Jul 2, 2025
06:28
E-Patient Counseling Trial E-PACO: Computer Based Education versus Nurse Counseling for Patients to Prepare for Colonoscopy
Published on: August 1, 2019
8.3K
ChatGPT vs. web search for patient questions: what does ChatGPT do better?
Sarek A Shen1, Carlos A Perez-Heydrich2, Deborah X Xie3
1Department of Otolaryngology-Head and Neck Surgery, Johns Hopkins School of Medicine, 601 North Caroline Street, Baltimore, MD, 21287, USA. sarek.shen@gmail.com.
Summary
ChatGPT provides more understandable medical information for diagnoses than web searches. Prompting improves ChatGPT readability for patients seeking health information online.
Area of Science:
- Medical Informatics
- Artificial Intelligence in Healthcare
- Patient Education
Background:
- Online patient information acquisition is evolving.
- Chat generative pretrained transformer (ChatGPT) offers new avenues for accessing medical data.
- Evaluating AI-generated health information against traditional sources is crucial.
Purpose of the Study:
- To compare the readability and appropriateness of ChatGPT responses versus traditional web searches for patient medical questions.
- To assess the accuracy and understandability of information provided by ChatGPT.
- To characterize the performance of ChatGPT across different question types (fact, policy, diagnosis).
Main Methods:
- Patient questions were sourced from online posts related to American Academy of Otolaryngology-Head and Neck Surgery guidelines.
- Questions were categorized into fact, policy, and diagnosis.
- ChatGPT and traditional web searches were used to answer questions.
- Readability (Flesch Reading Ease, Flesch-Kinkaid Grade Level) and understandability (PEMAT) were assessed.
- Accuracy was evaluated by blinded clinical experts.
Main Results:
- ChatGPT responses had lower average readability than web searches (FRE: 42.3 vs. 55.6).
- Understandability was comparable between ChatGPT and web search (PEMAT: 93.8% vs. 93.5%).
- ChatGPT outperformed web search for diagnosis-related questions (p < 0.01).
- Readability of ChatGPT responses improved with additional prompting (FRE 55.6).
Conclusions:
- ChatGPT is superior to web search for symptom-based diagnostic questions and equivalent for factual and policy information.
- Targeted prompting can enhance the readability of ChatGPT responses without compromising accuracy.
- Educating patients on the benefits and limitations of AI tools for medical information is essential.
Related Concept Videos
01:21Methods of Documentation III: PIE
1.4K
Problem-intervention-evaluation (PIE) is a systematic approach to documentation used in healthcare settings for clinical decision-making and patient care planning. It is a structured approach to organizing patient data based on problems, interventions, and evaluations. Here's a breakdown of its key features and considerations:
1.4K
01:13Patient-centered Care
2.0K
Patient-centered care involves delivering care beyond inpatient hospitalization. Reflective practice can enhance a patient-centered approach. Reflective practice is a process of reasoning that considers all aspects of the present situation, including practicalities, learning from personal practice, and consideration of patient needs. Patients appreciate care decisions made while considering their input. Involving the patient in their care provides the patient with a sense of contribution rather...
2.0K
01:26Methods of Documentation II: POMR
950
The Problem-Oriented Medical Record (POMR) revolutionized medical record-keeping by introducing a systematic approach focusing on the patient's problems rather than merely listing symptoms. Dr. Lawrence Weed's introduction of this method in the 1960s marked a significant advancement in medical documentation. The POMR framework consists of four key components: the database, problem list, plan of care, and progress notes.
950
01:15Methods of Documentation VI: Case Management Model
573
The case management model is a multidisciplinary approach that involves healthcare professionals from diverse disciplines, such as physicians, nurses, therapists, social workers, and pharmacists, working collaboratively to address the various needs of patients. Each healthcare professional brings unique expertise and perspectives, contributing to a more comprehensive understanding of the patient's condition and tailoring treatment plans accordingly.
For example, a patient with a chronic...
For example, a patient with a chronic...
573
01:29Assessment of the Gastrointestinal System II: Health Perception Pattern
114
Assessing the gastrointestinal (GI) system is a complex process that begins with collecting subjective data. This data, collected through patient interviews, provides crucial insights into the patient's health history, perception patterns, and lifestyle habits, all contributing significantly to GI health.
Health Perception Patterns
Health perception patterns offer valuable insights into a patient's lifestyle habits and how they may impact their GI health. These patterns include:
Health Perception Patterns
Health perception patterns offer valuable insights into a patient's lifestyle habits and how they may impact their GI health. These patterns include:
114
01:29Issues And Trends In Healthcare Delivery System
5.6K
The issues and trends in healthcare delivery are constantly changing. The COVID-19 pandemic is one recent issue that wreaked havoc on healthcare systems, causing a shortage of healthcare workers, high demand for medicines and supplies, and increased medical expenditure due to a lack of insurance. Other issues include rising healthcare costs and care fragmentation.
Cost Containment
Payment for healthcare services has historically promoted adoption of costly and often unnecessary or inefficient...
Cost Containment
Payment for healthcare services has historically promoted adoption of costly and often unnecessary or inefficient...
5.6K