Evaluating Algorithmic Bias in 30-Day Hospital Readmission Models: Retrospective Analysis

Area of Science:

• Health Informatics
• Health Equity
• Predictive Modeling

Background:

• Predictive algorithms in healthcare risk exacerbating existing disparities due to algorithmic bias.
• Existing fairness metrics for bias measurement have limited real-world application.

Purpose of the Study:

• To evaluate algorithmic bias in common 30-day hospital readmission models.
• To assess the utility and interpretability of selected fairness metrics.

Main Methods:

• Retrospective analysis of 10.6 million inpatient discharges (Maryland and Florida, 2016-2019).
• Evaluated LACE Index, modified HOSPITAL score, and modified CMS readmission measure (as-is and retrained).
• Assessed predictive performance and bias (false negative rate, false positive rate, 0-1 loss, generalized entropy index) across racial and income groups.

Main Results:

• Retrained CMS model showed best predictive performance; modified HOSPITAL score had best calibration.
• Calibration favored White and higher-income groups; AUC was higher/similar in Black populations.
• Retrained CMS and modified HOSPITAL score exhibited lowest bias in Maryland; modified HOSPITAL score showed lowest racial bias in Florida.
• Higher false negative rates observed in White/higher-income groups; higher false positive rates and 0-1 loss in Black/low-income groups.
• Models demonstrated heterogeneous algorithmic bias across different contexts and populations.

Conclusions:

• Fairness metrics can detect disparate model performance but require cautious interpretation.
• Statistical bias measures alone may obscure root causes of health disparities.
• Addressing bias necessitates considering imperfect data, analytic frameworks, and health systems; fairness metrics are a crucial first step.