Natural Language Processing to Automate Cerebrovascular Event Identification in Stroke Alerts.

Area of Science:

• Clinical informatics
• Machine learning in healthcare
• Cerebrovascular disease research

Background:

• Frequent false-positive stroke alerts strain healthcare resources.
• Machine learning models can predict stroke alert accuracy but require extensive manual data labeling.
• Weak labeling offers a method to accelerate machine learning development using heuristic rules.

Purpose of the Study:

• To label a large, unlabeled sample of stroke alerts using weak labeling.
• To determine the binary outcome (presence/absence of acute cerebrovascular disease) for stroke alerts.
• To evaluate the performance of a weak labeling framework for stroke alerts.

Main Methods:

• Developed a 4-tier hierarchical weak labeling heuristic ensemble.
• Utilized rule-based named-entity recognition on radiology reports (Tier 1).
• Aggregated outputs, incorporated diagnosis codes, and combined labels for a final encounter-level outcome (Tiers 2-4).

Main Results:

• Analyzed 16,512 stroke alert activations (2011-2021).
• Tier 1 (reports) achieved 0.84 sensitivity, 0.96 specificity, 0.87 F1.
• Tier 4 (final label) achieved 0.92 sensitivity, 0.86 specificity, 0.87 F1.

Conclusions:

• Successfully labeled a large registry of stroke alerts using weak labeling.
• The developed framework demonstrates potential for efficient clinical data annotation.
• This approach can be extended to other clinical datasets for machine learning applications.