Related Experiment Videos
Transient Ischemic Attack Diagnosis using Explicit Criteria Outperforms Artificial Intelligence: a Prospective Cohort
Background:
Diagnosing transient ischemic attacks (TIAs) remains challenging, particularly in primary care and emergency department settings where specialist neurologist consultation may be limited. Recently developed Explicit Diagnostic Criteria for TIA (EDCT) aim to guide diagnosis, while artificial intelligence (AI)-based diagnostic tools have also emerged, but their comparative effectiveness in clinical practice is uncertain.
Methods:
We conducted a prospective study of 127 patients referred to a specialist stroke service from an emergency department between June 2023 and February 2024 with suspected TIA. The diagnostic performance of three EDCT versions was compared against an advanced AI model (OpenAI O3-mini-high) using real-world clinical data. Final diagnoses by stroke neurologists served as the reference standard.
Results:
Of 127 patients, 67 (52.8%) had a final diagnosis of TIA or minor stroke. EDCT Versions 1, 2, and 3 demonstrated sensitivities of 95.5%, 91.0%, and 91.0% respectively, and specificities of 23.3%, 43.3%, and 46.7%. The AI model had a sensitivity of 98.5% but a very low specificity (11.7%; 95% CI 5.8-22.2). Steering improved AI specificity slightly (18.3%) but at the cost of decreased sensitivity.
Conclusion:
Structured clinical criteria (EDCT) outperformed an advanced AI model in distinguishing TIA from mimics in an emergency setting. AI classified almost all patients (119 out of 127) as having had a TIA, resulting in high sensitivity but very poor specificity, highlighting significant limitations when applied to real-world clinical scenarios. Future development of AI diagnostic tools should include rigorous validation against structured clinical standards.
