Fair admission risk prediction with proportional multicalibration

Area of Science:

• Machine Learning
• Medical Informatics
• Fairness in AI

Background:

• Fair calibration is crucial for trustworthy risk prediction models.
• Multicalibration ensures overall and subpopulation calibration but can lead to disparate error rates.
• Existing methods may allow decision-makers to unfairly trust or distrust predictions for specific groups.

Purpose of the Study:

• To propose proportional multicalibration (PMC) as a novel fairness criterion.
• To ensure calibration error is constrained across groups and within prediction bins.
• To improve model trustworthiness by limiting performance distinctions between patient groups.

Main Methods:

• Developed the proportional multicalibration (PMC) criterion.
• Proved that PMC bounds multicalibration and differential calibration.
• Created an efficient post-processing algorithm for achieving PMC.

Main Results:

• Proportionally multicalibrated models limit decision-makers' ability to distinguish performance across groups.
• Empirical evaluation and simulations demonstrate PMC's effectiveness.
• PMC effectively controls simultaneous measures of calibration fairness across intersectional groups.

Conclusions:

• Proportional multicalibration is a promising criterion for enhancing fairness in risk prediction.
• PMC offers a method to improve model trustworthiness with minimal impact on classification performance.
• The proposed algorithm provides a practical approach for implementing PMC in real-world applications.