Identifying Early Signals From Emerging Public Health Events Using Natural Language Processing.

Area of Science:

• Public Health
• Infectious Disease Surveillance
• Computational Epidemiology

Background:

• Emerging public health threats pose challenges due to surveillance systems not being optimized for novel events.
• Identifying common early signals across diverse emerging threats is crucial for timely detection.

Purpose of the Study:

• To identify and evaluate nonspecific early signals for emerging public health threats.
• To assess the utility of public health authority communication, zoonotic exposure, and other pathogen exposure mentions as early indicators.

Main Methods:

• Utilized U.S. Department of Veterans Affairs emergency department visit data (2004-2024).
• Employed rule-based and transformer models for natural language processing to extract target concepts from over 33 million clinical documents.
• Constructed training and validation sets based on historical diagnoses and laboratory data for selected infectious diseases.

Main Results:

• Positive predictive values for extracted signals ranged from 0.615 to 1.0.
• Extracted exposure distributions aligned with expectations for identified pathogens.
• Demonstrated successful extraction of relevant cases from historical clinical documents.

Conclusions:

• Automated natural language processing enables scalable surveillance of clinical documents for identifying relevant cases.
• Initial validation indicates acceptable accuracy for feasibility exploration in biosurveillance.
• Nonspecific early signals show promise for enhancing the detection of emerging public health threats.