So You've Got a High AUC, Now What? An Overview of Important Considerations when Bringing Machine-Learning Models...

Area of Science:

• Clinical Informatics
• Artificial Intelligence in Medicine
• Health Data Science

Background:

• Machine-learning (ML) models offer transformative potential for personalized, data-driven clinical decision-making.
• Successful integration of ML into clinical practice necessitates addressing factors beyond mere predictive accuracy.

Purpose of the Study:

• To provide a comprehensive guide for clinicians on developing and implementing clinically applicable machine-learning classification models.
• To highlight essential considerations for bridging the gap between ML model development and real-world healthcare applications.

Main Methods:

• Overview of methods for assessing and improving model calibration.
• Discussion on selecting appropriate decision thresholds and enhancing model explainability.
• Strategies for identifying and mitigating bias, robust validation, and real-world testing.

Main Results:

• Clinicians gain insights into crucial aspects of ML model deployment, including calibration, thresholding, explainability, and bias.
• Guidance is provided on robust validation techniques and strategies for real-world testing of ML models.
• The tutorial emphasizes the importance of accessibility and practical application of ML tools in healthcare.

Conclusions:

• Adherence to the presented guidance is crucial for the effective clinical deployment of machine-learning models.
• Addressing calibration, explainability, bias, and validation ensures ML models are reliable and trustworthy.
• Successful implementation of these ML models can significantly enhance patient care outcomes.