Value of diversity characteristics in predictive modeling: ACS screening as a case study

We sought to improve the subgroup performance variability of a model that identifies arriving ED patients at high risk for ACS, to receive an ECG within 10 minutes of arrival, to detect STEMI. We compared a Base Model using age, sex, and chief complaint alone to (1) an Interactions Model adding interactions among the 3 variables, and (2) a Diversity-Sensitive Model including race, ethnicity, language, as well as identity interactions. We quantified human performance and combined it with each of the 3 models simulating use as practice augmenting AI predictions. With sensitivity as our primary outcome, we found humans at 72.8% were bested by the Diversity-Sensitive Model at 82.8%, and by the human-augmented Diversity-Sensitive Model at 91.3%, improving ACS predictions in all demographic subgroups. However, there was residual variation among subgroups (range of sensitivity: 62%-98%). Given risk distribution differences, subgroup-specific ECG-testing thresholds may further equitize ACS prediction performance.