Related Experiment Video
Updated: Jun 12, 2025
A Simplified Stepwise Approach to Echo Guidance during Percutaneous Mitral Valve Repair
Published on: October 16, 2021
Evaluating Performance of ChatGPT on MKSAP Cardiology Board Review Questions
Stefan Milutinovic1, Marija Petrovic2, Dustin Begosh-Mayne1
1Florida State University College of Medicine Internal Medicine Residency Program at Lee Health, Cape Coral, Florida, USA.
ChatGPT demonstrated moderate performance on cardiovascular medicine board-style questions, comparable to residents but below cardiology attendings. Further AI development is needed for reliable medical education tools.
Area of Science:
- Artificial Intelligence in Medicine
- Medical Education Technology
- Natural Language Processing
Background:
- Artificial intelligence (AI) and natural language processing tools like ChatGPT show potential for enhancing medical practice.
- Discussion is ongoing regarding AI's role in improving efficiency and reducing errors in healthcare.
Purpose of the Study:
- To evaluate the performance of ChatGPT (versions 3.5 and 4) in answering cardiovascular medicine board-style questions.
- To compare AI performance against internal medicine residents and attending physicians.
Main Methods:
- Ninety-eight multiple-choice questions from the Cardiovascular Medicine Chapter of the Medical Knowledge Self-Assessment Program (MKSAP) were used.
- Performance of ChatGPT-3.5, ChatGPT-4, internal medicine interns, senior residents, internal medicine attendings, and cardiology attendings was assessed.
Main Results:
- ChatGPT-4 achieved 74.5% accuracy, comparable to residents and internal medicine attendings, but lower than cardiology attendings (85.7%).
- No significant difference was found between ChatGPT and physicians in most subcategories, except for valvular heart disease and heart failure.
- ChatGPT-4 outperformed senior residents in heart failure questions.
Conclusions:
- ChatGPT shows promise as an educational tool but requires improved accuracy to surpass instructors.
- AI performance needs to approach near-perfect scores to be considered a reliable tool for medical professionals.
More Related Videos
10:17Real-Time Cardiac Mapping with a Noninvasive Imageless Electrocardiographic Imaging System
Published on: April 11, 2025
12:45Benefits of Cardiac Resynchronization Therapy in an Asynchronous Heart Failure Model Induced by Left Bundle Branch Ablation and Rapid Pacing
Published on: December 11, 2017
Related Concept Videos
01:29Electrocardiogram
Three major waveforms are present in a typical ECG recording: the P wave, the QRS complex, and...
01:27Assessment of the Cardiovascular System III: Palpation
Jugular Venous Pressure (JVP) Measurement
Position the patient at a thirty- to forty-five-degree angle or in a semi-fowler's position. Look for the highest point of pulsation in the internal jugular vein and measure the vertical distance to the angle of Loius or sternal angle. A normal JVP is 3-4 cm above...
01:23Assessment of the Cardiovascular System I: Subjective Data
Initial Enquiry
Ask the patient about their primary concern and thoroughly explore all reported symptoms.
Medical History
Investigate past illnesses affecting the cardiovascular system, such as angina, anemia, rheumatic fever, congenital heart disease, stroke, thrombophlebitis, dysrhythmias, varicosities
Inquire about symptoms...