Related Experiment Video
Updated: May 8, 2025
Project-Based Learning Guidelines for Health Sciences Students: An Analysis with Data Mining and Qualitative Techniques
Published on: December 9, 2022
Comparing ChatGPT-4 and a Paediatric Intensive Care Specialist in Responding to Medical Education Questions: A
Shai Yitzhaki1,2, Nadav Peled3, Eytan Kaplan2,4
1Department of Pediatrics A, Schneider Children's Medical Center of Israel, Petach Tikva, Israel.
Objective:
To compare the performance of the Generative Pre-trained Transformer model 4 (ChatGPT-4) with that of a paediatric intensive care unit (PICU) specialist in responding to open-ended medical education questions.
Methods:
A comparative analysis was conducted using 100 educational questions sourced from a PICU trainee WhatsApp forum, covering factual knowledge and clinical reasoning. Ten PICU specialists from multiple tertiary paediatric centres independently evaluated 20 sets of paired responses from ChatGPT-4 and a PICU specialist (the original respondent to the forum questions), assessing overall superiority, completeness, accuracy, and integration potential.
Results:
After excluding one question requiring a visual aid, 198 paired evaluations were made (96 factual knowledge and 102 clinical reasoning). ChatGPT-4's responses were significantly longer than those of the PICU specialist (median words: 189 vs. 41; p < 0.0001). ChatGPT-4 was preferred in 60% of factual knowledge comparisons (p < 0.001), while the PICU specialist's responses were preferred in 67% of clinical reasoning comparisons (p < 0.0001). ChatGPT-4 demonstrated superior completeness in factual knowledge (p = 0.02) but lower accuracy in clinical reasoning (p < 0.0001). Integration of both answers was favoured in 37% of cases (95% CI, 31%-44%).
Conclusions:
ChatGPT-4 shows promise as a tool for factual medical education in the PICU, excelling in completeness. However, it requires oversight in clinical reasoning tasks, where the PICU specialist's responses remain superior. Expert review is essential before using ChatGPT-4 independently in PICU education and in other similarly underexplored medical fields.
More Related Videos
05:04Author Spotlight: Evaluating Clinicians' Adoption of Ultrasound-Guided Vascular Cannulation Through Simulation Training
Published on: August 9, 2024
06:28E-Patient Counseling Trial E-PACO: Computer Based Education versus Nurse Counseling for Patients to Prepare for Colonoscopy
Published on: August 1, 2019
Related Concept Videos
01:21Methods of Documentation III: PIE
01:18Interdisciplinary Care: The Health Care Team-II
Physical Therapist
A physical therapist (PT) aims to restore function or prevent additional impairment in a patient following an injury or disease. Massage, heat, cold, water, sonar waves, exercises, and electrical stimulation are some treatments used by PTs to treat...
01:26Methods of Documentation II: POMR
01:14SBAR II: Application of SBAR
SBAR Report from a Nurse to a Health Care Provider
S: "Hello, Dr. Smith. This is Jane, RN, from the Med Surg unit. I am calling to tell you about Ms. White in Room 210, who is experiencing increased pain and redness at her incision site. Her recent...
01:27Parentral Nutrition: Centeral and Peripheral Parental Nutrition
PN can be administered through two primary routes:
1. Central Parenteral Nutrition (CPN):
CPN involves delivering a high concentration of nutrients through a large vein. This is typically achieved using a Peripherally Inserted Central Catheter (PICC) or,...