Mitigated deployment strategy for ethical AI in clinical settings

Area of Science:

• Medical Artificial Intelligence (AI)
• Clinical Decision Support Systems
• Health Equity and Bias

Background:

• Clinical diagnostic tools can exhibit differential performance across demographic subgroups due to unrepresentative training data.
• Lack of established practical deployment solutions exacerbates harm to under-represented groups from AI models with performance disparities.
• Existing approaches, like selective deployment, risk excluding certain subgroups entirely.

Purpose of the Study:

• To introduce and evaluate a 'mitigated deployment' strategy for clinical AI models exhibiting differential performance.
• To explore the benefits and limitations of integrating safety nets within clinical workflows for universal AI deployment.
• To provide healthcare organizations with tools for the safe deployment of AI models across diverse patient populations.

Main Methods:

• Proposed a 'mitigated deployment' strategy involving human-AI collaboration and post-market evaluation.
• Utilized a real-world case study to analyze the practical application of the mitigated deployment approach.
• Focused on continuous improvement of model performance across subgroups using real-world data.

Main Results:

• The mitigated deployment strategy offers a viable alternative to selective deployment for addressing AI performance disparities.
• Human-AI collaboration and ongoing evaluation are crucial for safeguarding under-represented groups in universal AI deployment.
• The case study provided insights into the practical benefits and limitations of the proposed strategy.

Conclusions:

• Mitigated deployment, incorporating safety nets and continuous learning, enhances the safe and equitable use of clinical AI.
• This approach supports universal deployment while actively protecting vulnerable subgroups from AI-induced harm.
• Further research and implementation are needed to refine and validate this strategy in diverse clinical settings.