A Decision-Theoretic Perspective on Fairness in Clinical Predictive Models.

Area of Science:

• Medical Informatics
• Biostatistics
• Health Equity

Background:

• Fairness in statistical models, particularly clinical prediction models, is a significant concern.
• Current fairness methods often prioritize parity in model predictions across groups, potentially overlooking broader clinical implications.
• Patient outcomes are critical when assessing fairness in healthcare applications.

Purpose of the Study:

• To evaluate the fairness of a revised clinical prediction model (UTICalc) using both prediction-based and outcome-based metrics.
• To investigate the potential divergence between prediction-based and outcome-based fairness in clinical decision-making.
• To propose a decision-theoretic framework for assessing fairness based on patient outcomes.

Main Methods:

• Analysis of the UTICalc model, revised for improved racial fairness and tested on equal opportunity (true positive rate).
• Development of a decision-theoretic framework incorporating patient outcome utilities and model predictions.
• Construction of a decision tree to model the clinical decision process for urinary tract infection (UTI) in children.

Main Results:

• The revised UTICalc model demonstrated improved performance on a prediction-based fairness metric (equal opportunity).
• However, the model did not show improvement in an outcome-based fairness metric (expected utility parity).
• This indicates a potential divergence between prediction-based and outcome-based fairness.

Conclusions:

• Fairness evaluation in clinical prediction models should encompass both prediction performance and patient outcomes.
• Relying solely on prediction-based fairness metrics may not adequately address fairness concerns in clinical practice.
• A comprehensive approach integrating patient utilities is essential for robust fairness assessment in healthcare AI.