Sources of Bias in Clinical Artificial Intelligence and Applications in Rheumatology

Area of Science:

• Rheumatology
• Artificial Intelligence
• Health Equity

Background:

• Machine learning (ML) models in rheumatology are increasingly used but face significant challenges.
• Preexisting, technical, and emergent biases can limit the reliability and fairness of these models.
• The interaction between data constraints, design choices, and clinical workflows contributes to model limitations.

Purpose of the Study:

• To highlight the multifaceted nature of bias in rheumatology ML models.
• To emphasize that biases arise from the entire model lifecycle and clinical integration.
• To advocate for proactive strategies to mitigate bias and ensure equitable application of ML in rheumatology.

Main Methods:

• Analysis of bias sources across the machine learning model lifecycle.
• Examination of how optimization objectives can encode and perpetuate disparities.
• Consideration of real-world clinical workflows and their impact on model performance.

Main Results:

• Bias in rheumatology ML models stems from a complex interplay of factors, including data, design, and clinical integration.
• Optimization objectives can inadvertently embed patterns of care and access disparities.
• Aggregate performance metrics often mask critical subgroup failures, obscuring real-world inequities.

Conclusions:

• Addressing bias in rheumatology ML requires deliberate design choices throughout the model lifecycle.
• Transparency and sustained oversight are crucial for mitigating hidden subgroup failures.
• Proactive strategies are essential to ensure ML models promote, rather than hinder, equitable rheumatic disease care.