Value of diversity characteristics in predictive modeling: ACS screening as a case study

Area of Science:

• * Emergency Medicine
• * Artificial Intelligence in Healthcare
• * Cardiology

Background:

• * Accurate identification of patients at high risk for acute coronary syndrome (ACS) is crucial in the emergency department (ED).
• * Existing models may exhibit performance variability across different demographic subgroups.
• * Timely electrocardiogram (ECG) detection of ST-elevation myocardial infarction (STEMI) is critical for patient outcomes.

Purpose of the Study:

• * To improve the subgroup performance variability of a model identifying high-risk ED patients for timely ECG.
• * To compare a base model with an interactions model and a diversity-sensitive model, including demographic factors.
• * To evaluate the impact of augmenting AI predictions with human performance.

Main Methods:

• * Compared a base model (age, sex, chief complaint) with an interactions model and a diversity-sensitive model (including race, ethnicity, language, identity).
• * Quantified human performance and simulated its combination with each AI model.
• * Used sensitivity as the primary outcome measure for predicting ACS risk.

Main Results:

• * The diversity-sensitive model achieved 82.8% sensitivity, outperforming the base model.
• * Human-augmented diversity-sensitive model reached 91.3% sensitivity, improving ACS predictions across all subgroups.
• * Residual sensitivity variation among subgroups ranged from 62% to 98%.

Conclusions:

• * A diversity-sensitive AI model significantly improves ACS prediction accuracy in ED patients.
• * Augmenting AI with human performance further enhances prediction, but subgroup disparities persist.
• * Subgroup-specific ECG-testing thresholds may be necessary to further equitize ACS prediction performance.