Related Experiment Video
Updated: Sep 21, 2025
A Machine Learning Approach to Design an Efficient Selective Screening of Mild Cognitive Impairment
Published on: January 11, 2020
Algorithms identifying low-acuity emergency department visits: A review and validation study
Angela T Chen1,2,3, Madhavi Muralidharan2, Ari B Friedman2,3
1Health Care Management Department, The Wharton School, University of Pennsylvania, Philadelphia, Pennsylvania, USA.
Objective:
To characterize and validate the landscape of algorithms that use International Classification of Disease (ICD) codes to identify low-acuity emergency department (ED) visits.
Data Sources:
Publicly available ED data from the National Hospital Ambulatory Medical Care Survey (NHAMCS).
Study Design:
We systematically searched for studies that specify algorithms consisting of ICD codes that identify preventable or low-acuity ED visits. We classified ED visits in NHAMCS according to these algorithms and compared agreements using the Jaccard index. We then evaluated the performance of each algorithm using positive predictive value (PPV) and sensitivity, with the reference group specified using low-acuity composite (LAC) criteria consisting of both triage and clinical components. In sensitivity analyses, we repeated our primary analysis using only triage or only clinical criteria for reference.
Data Collection:
We used the 2011-2017 NHAMCS data, totaling 163,576 observations before survey weighting and after dropping observations missing a primary diagnosis. We translated ICD-9 codes (years 2011-2015) to ICD-10 using a standard crosswalk.
Principal Findings:
We identified 15 papers with an original list of ICD codes used to identify preventable or low-acuity ED presentations. These papers were published between 1992 and 2020, cited an average of 310 (SD 360) times, and included 968 (SD 1175) codes. Pairwise Jaccard similarity indices (0 = no overlap, 1 = perfect congruence) ranged from 0.01 to 0.82, with mean 0.20 (SD 0.13). When validated against the LAC reference group, the algorithms had an average PPV of 0.308 (95% CI [0.253, 0.364]) and sensitivity of 0.183 (95% CI [0.111, 0.256]). Overall, 2.1% of visits identified as low acuity by the algorithms died prehospital or in the ED, or needed surgery, critical care, or cardiac catheterization.
Conclusions:
Existing algorithms that identify low-acuity ED visits lack congruence and are imperfect predictors of visit acuity.
