Accuracy of GPT-5 and GPT-4o in diagnosing STEMI from 12-Lead ECGs: A comparative study with cardiologists and
Kamil Kokulu1, Muhammed Akay2, Ekrem Taha Sert1
1Department of Emergency Medicine, Aksaray Training and Research Hospital, Aksaray, Turkey; Department of Emergency Medicine, Aksaray University School of Medicine, Aksaray, Turkey.
Background:
ST-segment elevation myocardial infarction (STEMI) requires rapid, accurate electrocardiogram (ECG) interpretation. The diagnostic effectiveness of new large language models (LLMs) like GPT-5 and GPT-4o in this high-risk area remains a critical knowledge gap.
Objectives:
We aimed to evaluate the diagnostic performance of GPT-5 and GPT-4o in diagnosing STEMI from 12-lead ECGs, comparing them against emergency medicine specialists (EMSs) and cardiologists.
Methods:
In a case-control study of 234 patients, we included 117 angiography-confirmed STEMI cases and 117 age- and sex-matched controls presenting with chest pain but without STEMI. Anonymized ECG images were presented to 3 EMSs, 3 cardiologists, GPT-5, and GPT-4o with a dichotomous (yes/no) question: "Is there a STEMI?"AI models were queried three times on different days to measure response consistency. Performance was compared using accuracy, sensitivity, specificity, predictive values, and likelihood ratios.
Results:
Cardiologists (Accuracy: 89.6%) and EMSs (Accuracy: 87.8%) significantly outperformed both GPT-5 (Accuracy: 69.9%) and GPT-4o (Accuracy: 55.9%) (p<0.001). While GPT-5's sensitivity (85.5%) was statistically comparable to clinicians (86.9%-88.6%), it exhibited a critically high false-positive (overcall) rate (45.6%) compared to cardiologists (7.7%) and EMSs (13.1%). GPT-4o's sensitivity was significantly lower (76.9%). GPT-5 showed substantial response consistency (Fleiss' Kappa=0.76), while GPT-4o's was fair (Fleiss' Kappa=0.26).
Conclusion:
GPT-5 approached clinician-level sensitivity but was unreliable due to an extremely high false-positive rate. GPT-4o's performance was poor. Current LLMs are not reliable as independent diagnostic tools for STEMI.
Related Concept Videos
01:29Electrocardiogram
Three major waveforms are present in a typical ECG recording: the P wave, the QRS complex, and...
01:28Electrocardiogram Fundamentals
An electrocardiogram (ECG) is a diagnostic tool for identifying cardiac conditions such as arrhythmias, conduction abnormalities, and myocardial ischemia.
Definition
An electrocardiogram (ECG) visualizes the heart's electrical activity by tracing the electrical movement associated with each heartbeat on a graph or monitor. As the heart beats, an electrical wave passes through it, correlating with the cardiac cycle events.
Parts of an ECG
An ECG utilizes electrodes on the skin...
01:30Acute Coronary Syndrome I: Introduction
01:30Acute Coronary Syndrome III: Diagnostic Studies
01:24ECG Interpretation of Rhythms
Components of the Electrocardiogram
The primary components of a normal ECG waveform in Normal sinus rhythm(NSR) include the P wave, PR interval, QRS complex, ST segment, T wave, and occasionally a U wave.
ECG waveforms are divided by vertical and horizontal lines at standard intervals.
The horizontal axis measures time and rate, and the vertical axis measures amplitude or voltage....
01:26Exercise Stress Test
Exercise stress testing, commonly known as a treadmill test, is a noninvasive procedure used to evaluate cardiovascular function and diagnose heart conditions.
Definition
An exercise stress test measures the heart's response to exertion using a treadmill or stationary bicycle. Chest electrodes record the heart's electrical activity through an ECG, and blood pressure is monitored regularly.
Purposes