Operationalizing AI in Stroke Alerts: Balancing Sensitivity and Specificity in Predicting Acute Cerebrovascular Disease

Margaret H Downes ¹, Asala N Erekat ^2,3, Benjamin R Kummer ^2,3,4,5

⁰Icahn School of Medicine at Mount Sinai, New York, New York, United States.

Seminars in Neurology +

|

December 11, 2025

View abstract on PubMed

Summary

A new AI model, ScanNER v2, helps identify acute cerebrovascular disease (ACD) in stroke alerts, reducing false positives. This machine learning tool aims to improve timely and accurate acute stroke care.

Area Of Science

Neurology
Artificial Intelligence
Health Informatics

Background

False positive acute stroke alerts are common, leading to diagnostic challenges.
Existing systems struggle to efficiently differentiate true acute cerebrovascular disease (ACD) from other conditions.

Purpose Of The Study

To describe the development and operationalization of an AI-enabled pipeline, ScanNER v2, for predicting ACD.
To outline a practical framework for integrating AI tools into hospital stroke systems.

Main Methods

Developed ScanNER v2, a machine learning model using large-language models and structured clinical data.
Validated the model on approximately 16,000 stroke alerts over 10 years.
Explored implementation strategies focusing on high sensitivity versus high specificity.

Main Results

ScanNER v2 achieved an area under the receiver-operating curve of 0.72 and an F1 score of 0.72.
The model demonstrated an overall positive predictive value of 0.68 for detecting ACD.
Identified operational and clinical tradeoffs for different implementation approaches.

Conclusions

AI-assisted tools like ScanNER v2 show promise in enhancing acute stroke care.
Responsible deployment requires careful consideration of implementation, governance, workflow, and equity.
Thoughtful integration of AI is crucial for improving the efficiency and accuracy of stroke alert systems.