Natural Language Processing to Identify Infants Aged 90 Days and Younger With Fevers Prior to Presentation

Natural language processing (NLP) can enhance research studies for febrile infants by more comprehensive cohort identification. We aimed to refine and validate an NLP algorithm to identify and extract quantified temperature measurements from infants aged 90 days and younger with fevers at home or clinics prior to emergency department (ED) visits.

We conducted a cross-sectional study using electronic health record (EHR) data from 17 EDs in 10 health systems that are part of the Pediatric Emergency Care Applied Research Network Registry. All visits between January 1, 2012, and May 31, 2023, for infants aged 90 days and younger were eligible, excluding those with trauma-related diagnoses. We iteratively refined a prespecified rules-based NLP algorithm in 7 successive samples of 200 visits and validated the algorithm on a held-out sample of 500 visits. The reference standard for pre-ED quantified temperature measurements was a temperature documented in clinical notes, excluding ED vital sign temperatures.

In our final sample, 113 of 500 visits (23%) had quantified temperature measurements. The NLP algorithm had sensitivity 95% (95% CI: 88%-98%), specificity 96% (95% CI: 93%-97%), and positive predictive value 86% (95% CI: 78%-91%). When applying rules to exclude temperatures that may have been noted more than 24 hours previously, the NLP algorithm had lower sensitivity (88%; 95% CI: 81%-93%) but similar specificity (97%; 95% CI: 95%-98%).

This highly accurate NLP algorithm can identify febrile infants without documented fevers in the ED to facilitate their inclusion in large studies using EHR data.