Artificial intelligence (ChatGPT 4.0) vs. Human expertise for epileptic seizure and epilepsy diagnosis and...

Artificial intelligence (AI) tools like ChatGPT hold promise for enhancing diagnostic accuracy and efficiency in clinical practice. This exploratory study evaluates ChatGPT's performance in diagnosing and classifying epileptic seizures, epilepsy, and underlying etiologies in adult patients compared to epileptologists and neurologists.

A prospective simulation study assessed 37 clinical vignettes based on real adult patient cases. ChatGPT was 'trained' using official ILAE documents on epilepsy diagnosis and classification. Diagnoses and classifications by ChatGPT, two epileptologists, and two neurologists were compared against a reference standard set by a senior epileptologist. Diagnostic accuracy was evaluated using sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV). Cohen's kappa (κ) was calculated to assess agreement.

ChatGPT demonstrated high sensitivity (≥96.9 %) in identifying epileptic seizures and diagnosing epilepsy, ensuring no cases were missed. However, its specificity was lower, particularly for distinguishing acute symptomatic from unprovoked seizures (33.3 %) and diagnosing epilepsy (26.7 %), leading to frequent false positives. ChatGPT excelled in diagnosing epileptic syndromes (κ = 1.00) and structural etiologies (accuracy = 90.0 %) but struggled with ambiguous cases such as unknown seizure onset (accuracy = 12.5 %) and rare etiologies. Human experts consistently outperformed ChatGPT with near-perfect accuracy and higher κ values.

ChatGPT shows potential as a supplementary diagnostic tool but requires human oversight due to reduced specificity and limitations in nuanced clinical judgment. Further development with diverse datasets and targeted training is necessary to improve AI performance. Integrating AI with expert clinicians can optimize diagnostic workflows in epilepsy care.