Mitigating Bias in Radiology Machine Learning: 3. Performance Metrics

Area of Science:

• Medical Artificial Intelligence
• Machine Learning in Healthcare
• Radiology Imaging Analysis

Background:

• Machine learning (ML) algorithms are increasingly used in clinical settings, raising concerns about potential bias.
• Bias can be introduced during data handling, model development, or performance evaluation of ML models.
• Addressing bias is crucial for the reliable implementation of ML in healthcare.

Purpose of the Study:

• To focus on the performance evaluation stage of ML model creation.
• To discuss strategies for mitigating and detecting bias in radiology artificial intelligence (AI) models.
• To highlight the importance of correct implementation of ML steps to minimize bias.

Main Methods:

• Reviewing performance evaluation toolboxes for ML models.
• Discussing model fitness, performance metrics, performance interpretation maps, and uncertainty quantification.
• Analyzing the strengths and limitations of various performance evaluation tools.

Main Results:

• Performance evaluation is a critical step for identifying and minimizing bias in ML models.
• Specific toolboxes like performance metrics and uncertainty quantification aid in bias detection.
• Understanding the limitations of these tools is key to effective bias mitigation.

Conclusions:

• Implementing correct performance evaluation strategies is essential for reducing bias in clinical ML models.
• Radiology AI models require careful evaluation to ensure fairness and accuracy.
• Further research into bias mitigation techniques in ML is warranted.