Counterfactual fairness for small subgroups.

Area of Science:

• Health Informatics
• Biostatistics
• Machine Learning Ethics

Background:

• Existing fairness metrics for risk prediction models struggle with small, marginalized subgroups.
• Clinical applications require fairness assessments that account for treatment confounding.
• Limitations in sample size hinder the redress of discrimination against vulnerable populations.

Purpose of the Study:

• To develop novel methods for assessing and correcting differential performance in risk prediction models for small subgroups.
• To address statistical challenges in clinical applications of risk prediction models.
• To enhance algorithmic fairness for marginalized groups in healthcare.

Main Methods:

• Proposed new estimands leveraging information across multiple groups.
• Estimated fairness quantities using a larger data volume than conventional techniques.
• Introduced a novel data borrowing approach using external data lacking outcomes.

Main Results:

• The developed methods allow for fairness assessments in smaller subgroups.
• The approach effectively incorporates external data to improve estimation.
• Demonstrated application on a real-world risk prediction model used during the COVID-19 pandemic.

Conclusions:

• The proposed 3-step approach enhances the ability to achieve algorithmic fairness in clinical risk prediction.
• This methodology addresses critical limitations of existing techniques, particularly for vulnerable populations.
• The findings have significant implications for equitable healthcare delivery and treatment guidance.