Addressing bias in prediction models by improving subpopulation calibration.

Area of Science:

• Medical prediction modeling
• Health equity research
• Biostatistics

Background:

• Medical prediction models, such as the Pooled Cohort Equations (PCE) and the fracture risk assessment tool (FRAX), are widely used but often exhibit poor calibration in subpopulations.
• This subpopulation miscalibration leads to biased risk predictions and reduced model performance for underrepresented groups, compromising healthcare equity.

Purpose of the Study:

• To address subpopulation miscalibration in predictive models.
• To adapt and validate a fairness algorithm for recalibrating prediction models.
• To improve the fairness and accuracy of risk assessments across diverse patient groups.

Main Methods:

• A retrospective cohort study evaluated the calibration of PCE and FRAX models in overall and specific subpopulations.
• An adapted fairness algorithm was applied to recalibrate predictions.
• Calibration metrics, including calibration-in-the-large, were assessed before and after recalibration.

Main Results:

• Baseline calibration was good overall but poor in many subpopulations.
• After applying the recalibration algorithm, subpopulation calibration significantly improved.
• The variance of calibration-in-the-large values decreased by 98.8% for PCE and 94.3% for FRAX models.

Conclusions:

• A model-independent, postprocessing fairness algorithm effectively reduces subpopulation miscalibration.
• Recalibration enhances fairness and equality in medical prediction models.
• This approach is crucial for ensuring reliable risk predictions for all patient groups.