Performance Metrics for the Comparative Analysis of Clinical Risk Prediction Models Employing Machine Learning.

Area of Science:

• Clinical decision-making
• Health informatics
• Predictive modeling

Background:

• Machine learning (ML) models are increasingly used to enhance clinical risk prediction.
• Commonly reported performance metrics may not adequately capture the advantages of ML models.
• ML models can improve risk estimation for specific subpopulations, often missed by standard metrics.

Purpose of the Study:

• To address limitations of current performance metrics for clinical risk prediction models.
• To propose additional metrics for comprehensive model evaluation.
• To facilitate the adoption of advanced models by healthcare professionals.

Main Methods:

• Review and discussion of performance metrics for overall performance, discrimination, calibration, resolution, and reclassification.
• Exploration of metrics related to model implementation.
• Illustration using models for predicting acute kidney injury after percutaneous coronary intervention.

Main Results:

• Standard performance metrics may lack the sensitivity to detect improvements offered by ML models.
• A comprehensive suite of metrics is crucial for accurate model comparison.
• Specific metrics can highlight improvements in risk estimation for subpopulations.

Conclusions:

• Commonly reported metrics are insufficient for evaluating advanced risk prediction models.
• A broader range of metrics is necessary for robust reporting and comparison.
• Adoption of a comprehensive metric set will aid healthcare professionals in leveraging ML for improved patient care.