Auditor models to suppress poor artificial intelligence predictions can improve human-artificial intelligence...

Area of Science:

• Healthcare AI
• Clinical Decision Support Systems
• Machine Learning Fairness

Background:

• Machine learning (ML) is increasingly used in healthcare, but can exhibit unfairness across subpopulations.
• Clinician overreliance on ML can perpetuate existing biases.
• ML suppression, which silences unreliable predictions, shows potential to mitigate these issues.

Purpose of the Study:

• To evaluate the impact of ML suppression on collaborative fairness between clinicians and AI.
• To assess ML uncertainty as a metric for auditing ML performance.

Main Methods:

• Utilized electronic health record data from Vanderbilt University Medical Center and MIMIC-IV-ED.
• Predicted patient outcomes (death, ICU transfer, 30-day readmission) using gradient-boosted trees and an oracle model.
• Simulated clinician acceptance of ML predictions and measured performance (AUC) and fairness (averaged odds difference).

Main Results:

• ML suppression outperformed human clinicians alone when ML performance was superior, without degrading fairness.
• When clinicians outperformed ML, suppression did not significantly degrade fairness.
• Integrating uncertainty quantification improved suppression-based approaches.

Conclusions:

• Auditing ML predictions via suppression shows promise for enhancing collaborative human-AI performance and fairness.
• Suppression effectively addresses issues arising from overreliance on suboptimal ML models.