Derivation of a natural language processing algorithm to identify febrile infants.

Area of Science:

• Pediatric Emergency Medicine
• Clinical Informatics
• Natural Language Processing

Background:

• Diagnostic codes have low sensitivity for identifying febrile infants, leading to underdetection in research samples.
• Ensuring representative study samples is crucial for accurate medical research findings.
• An improved method is needed to reliably identify febrile infants for clinical studies.

Purpose of the Study:

• To develop and validate a natural language processing (NLP) algorithm for identifying febrile infants.
• To compare the performance of the NLP algorithm against traditional diagnostic codes.
• To enhance the accuracy and representativeness of patient samples in pediatric research.

Main Methods:

• A cross-sectional study of infants aged 0-90 days was conducted in a pediatric emergency department.
• Two rule-based NLP algorithms were developed using clinical notes from 2017 and tested on 2016 data.
• Algorithm performance was compared to diagnostic codes using manual abstraction as the gold standard, measuring AUC, sensitivity, and specificity.

Main Results:

• The NLP algorithms (Models 1, 2) achieved significantly higher AUC (0.92-0.95) and sensitivity (86%-92%) compared to diagnostic codes (Models 5-8).
• Specificities for NLP algorithms were high (93%-99%), comparable to diagnostic codes.
• Samples identified by NLP algorithms mirrored gold standard characteristics, including fever prevalence and infection rates.

Conclusions:

• Rule-based NLP algorithms accurately identify febrile infants with superior sensitivity and comparable specificity to diagnostic codes.
• These algorithms can create more representative study samples, potentially improving the generalizability of research findings.
• External validation of these NLP tools is recommended for broader clinical application.