Pilot Study Comparing Emergency Physician and Artificial Intelligence-supported Interpretations of Electrocardiograms

Artificial intelligence (AI) tools are increasingly being explored for medical applications; however, their effectiveness in emergency electrocardiogram (ECG) interpretation is under-investigated. In this pilot study we aimed to evaluate the diagnostic performance of AI interpretation of ECGs by comparing its results to those of an experienced emergency physician.

We sourced 20 ECG cases representing common critical conditions from publicly available academic repositories-ST-elevation myocardial infarction, non-ST-elevation myocardial infarction, atrial fibrillation, supraventricular tachycardia, ventricular tachycardia, third-degree atrioventricular block, left and right bundle branch block, hyperkalemia, Brugada syndrome, Wellens syndrome, fusion beats, and torsades de pointes.The AI tool ChatGPT-4 and an experienced emergency physician, who served as the reference (gold standard) interpreter, independently analyzed each case across five key parameters: rhythm and heart rate; cardiac axis; ST/T segment changes; preliminary diagnosis; and emergency management recommendation. Full concordance was defined as complete agreement across all five parameters.

Agreement between AI and the emergency physician was observed in 18 of 20 cases (90%). The Cohen kappa was 0.80, indicating substantial chance-corrected agreement. Concordance by diagnostic category was as follows: myocardial infarction (6/6, 100%); arrhythmias including atrial fibrillation, supraventricular tachycardia, and ventricular tachycardia (6/6, 100%); conduction disorders (3/3, 100%); and hyperkalemia (1/1, 100%). Among the atypical or complex ECGs, concordance was 1/1 (100%) for Brugada syndrome, 1/1 (100%) for Wellens syndrome, 0/1 (0%) for fusion beats, and 0/1 (0%) for torsades de pointes. In the torsades case, ChatGPT did not recommend intravenous magnesium sulfate-the standard first-line treatment-despite recognizing the condition.

An AI tool demonstrated moderate diagnostic concordance with one experienced emergency physician in interpreting some common ECG findings in the emergency setting. However, discrepancies, particularly in complex cases and critical management recommendations, highlight the need for larger scale investigations. Our findings from this pilot study do not support the independent use of AI for definitive ECG interpretation or emergency management decisions; it should serve as an adjunct tool that enhances rather than supplants human clinical judgment.