Operationalizing AI in Stroke Alerts: Balancing Sensitivity and Specificity in Predicting Acute Cerebrovascular...

Area of Science:

• Neurology
• Artificial Intelligence
• Health Informatics

Background:

• False positive acute stroke alerts are common, leading to diagnostic challenges.
• Existing systems struggle to efficiently differentiate true acute cerebrovascular disease (ACD) from other conditions.

Purpose of the Study:

• To describe the development and operationalization of an AI-enabled pipeline, ScanNER v2, for predicting ACD.
• To outline a practical framework for integrating AI tools into hospital stroke systems.

Main Methods:

• Developed ScanNER v2, a machine learning model using large-language models and structured clinical data.
• Validated the model on approximately 16,000 stroke alerts over 10 years.
• Explored implementation strategies focusing on high sensitivity versus high specificity.

Main Results:

• ScanNER v2 achieved an area under the receiver-operating curve of 0.72 and an F1 score of 0.72.
• The model demonstrated an overall positive predictive value of 0.68 for detecting ACD.
• Identified operational and clinical tradeoffs for different implementation approaches.

Conclusions:

• AI-assisted tools like ScanNER v2 show promise in enhancing acute stroke care.
• Responsible deployment requires careful consideration of implementation, governance, workflow, and equity.
• Thoughtful integration of AI is crucial for improving the efficiency and accuracy of stroke alert systems.