AI assisted detection of large vessel occlusion on non-contrast CT: multinational validation and reader study

Early detection of large vessel occlusion (LVO) on non-contrast CT (NCCT) could accelerate stroke triage, but NCCT based artificial intelligence (AI) algorithms lack multinational validation and evidence of impact on clinician performance. We aimed to validate a machine learning based LVO detection algorithm across multinational cohorts and evaluate its impact on clinicians' diagnostic performance.

In this retrospective study, an AI algorithm was validated using independent cohorts from Korea (n=723; consecutive; 127 with LVO) and the US (n=240; case-control; 120 with LVO). Standalone performance was assessed using the area under the receiver operating characteristic curve (AUC), sensitivity, and specificity. A multi-reader, multi-case crossover study involving eight physicians evaluated diagnostic performance with and without AI assistance. Clinical utility was quantified using net reclassification improvement (NRI), number needed to screen (NNS), and benefit-to-harm ratio (BHR).

Standalone AI achieved AUC values of 0.963 (95% CI 0.946 to 0.975) in the Korean cohort and 0.899 (95% CI 0.858 to 0.939) in the US cohort. AI assistance significantly increased pooled AUC values from 0.718 to 0.852, sensitivity from 46.6% to 63.7%, and specificity from 91.9% to 94.9% (all P<0.001). AI gave an NRI of 5.5%, an NNS of 18.2, and a BHR of 2.89. Automation bias analysis showed a reliance level BHR of 9.25.

This multinational validation confirmed the robustness and generalizability of a machine learning based LVO detection algorithm on NCCT. AI assistance significantly improved clinicians' sensitivity and diagnostic accuracy, suggesting its potential as a supportive tool in acute stroke triage.