Evaluating sociodemographic bias in a deployed machine-learned patient deterioration model.

Area of Science:

• Clinical Informatics
• Artificial Intelligence in Healthcare
• Health Equity Research

Background:

• Machine learning (ML) model bias evaluations typically focus on research settings, neglecting clinical deployment impacts.
• Assessing downstream bias in real-world ML applications is crucial for equitable healthcare.
• The study addresses the need to evaluate algorithmic bias and care disparities in deployed ML systems.

Purpose of the Study:

• To evaluate the CHARTwatch early warning system for algorithmic bias in model performance.
• To assess disparities in care processes and outcomes across patient sociodemographic groups.
• To determine if CHARTwatch implementation impacts equity in inpatient care.

Main Methods:

• The study compared patient outcomes using propensity score overlap weighting before and after CHARTwatch implementation.
• Evaluations included model performance (sensitivity, specificity), care processes, and patient outcomes (in-hospital death).
• Differences were analyzed across sociodemographic subgroups: age, sex, homelessness, and neighborhood characteristics.

Main Results:

• CHARTwatch demonstrated consistent sensitivity across all subgroups.
• Model specificity varied by age, with lower specificity in older adults (>80 years).
• Implementation increased code status documentation for patients experiencing homelessness, with no other significant care process or outcome differences.

Conclusions:

• CHARTwatch model performance and impact were largely consistent across sociodemographic subgroups.
• Standardized care through ML tools may reduce existing inequities, as seen with code status orders for homeless patients.
• The study provides a framework for future bias assessments of deployed ML-CDS tools.