Identifying anaphylaxis using weakly-supervised prediction models and natural language processing

Area of Science:

• Health Informatics
• Computational Biology
• Clinical Research

Background:

• Scalable computable phenotyping algorithms are essential for high-throughput disease-outcome research using large electronic health record (EHR) and claims datasets.
• Anaphylaxis is a rare condition that presents diagnostic challenges when relying solely on claims data.

Purpose of the Study:

• To develop and evaluate a computable phenotyping algorithm for anaphylaxis using both EHR and claims data.
• To assess the performance of NLP-driven models in identifying anaphylaxis cases within healthcare systems.

Main Methods:

• Engineered features from clinical text using automated natural language processing (NLP).
• Developed a phenotyping algorithm utilizing four NLP- and diagnosis code-based silver labels as proxies for gold-standard labels.
• Evaluated algorithm performance against gold-standard abstracted outcomes from two healthcare systems (KPWA and VUMC).

Main Results:

• The top-performing NLP-based silver-label model achieved an area under the receiver operating characteristic curve (AUC) of 0.931 at KPWA.
• Positive predictive value (PPV) ranged from 0.52 to 0.77, and sensitivity ranged from 0.78 to 1, depending on the model and site.
• High sensitivity for anaphylaxis identification was achievable with the developed models.

Conclusions:

• NLP-based models demonstrated strong performance, particularly at KPWA, offering a better trade-off between PPV and sensitivity compared to prior manual methods.
• The algorithm's simplicity facilitates easy deployment across multiple healthcare systems for efficient phenotyping.
• The developed algorithm enhances the ability to conduct large-scale disease-outcome research for rare conditions like anaphylaxis.