Auditor Models to Suppress Poor AI Predictions Can Improve Human-AI Collaborative Performance

Area of Science:

• Artificial Intelligence in Healthcare
• Machine Learning Fairness
• Human-AI Collaboration

Background:

• Machine learning (ML) models are increasingly used in healthcare decision-making.
• ML models can exhibit unfairness, leading to inconsistent outcomes across patient subpopulations.
• Clinician overreliance on ML can perpetuate or exacerbate existing unfairness.

Purpose of the Study:

• To evaluate the impact of ML suppression on collaborative fairness between clinicians and AI.
• To assess the role of ML uncertainty quantification in auditing ML predictions.
• To improve the performance and fairness of human-AI collaboration in healthcare.

Main Methods:

• Utilized electronic health record data from Vanderbilt University Medical Center and MIMIC-IV-ED.
• Predicted patient outcomes (death, ICU transfer, 30-day readmission) using gradient-boosted trees and an oracle model.
• Simulated clinician decisions based on empirical data and measured performance using AUC and fairness using averaged odds difference.

Main Results:

• ML suppression improved collaborative performance when ML outperformed humans (p < 0.034) without degrading fairness.
• When humans outperformed ML, suppression improved performance (p < 5.2 × 10^-5), but human decision-making was fairer (p < 0.0019).
• Integrating uncertainty quantification into suppression methods enhanced overall performance.

Conclusions:

• Suppression of low-quality ML predictions via an auditor model shows potential for enhancing human-AI collaboration.
• This approach can improve both the performance and fairness of AI systems in clinical settings.
• Uncertainty quantification is a valuable tool for auditing ML and improving collaborative outcomes.