Weighted Brier Score - an Overall Summary Measure for Risk Prediction Models with Clinical Utility Consideration

Area of Science:

• Biostatistics
• Clinical Epidemiology
• Health Services Research

Background:

• Biomarker-based algorithms are increasingly used for disease risk prediction.
• Evaluating these models requires assessing their clinical utility beyond mere accuracy.
• The traditional Brier score is inadequate for measuring clinical utility.

Purpose of the Study:

• To propose a novel class of weighted Brier scores for assessing clinical utility in risk prediction.
• To align prediction model evaluation with decision-theoretic principles of clinical utility.
• To provide a more comprehensive assessment of risk prediction models in clinical practice.

Main Methods:

• Developed a class of weighted Brier scores based on decision theory.
• Decomposed the weighted Brier score into discrimination and calibration components.
• Established a theoretical link between the weighted Brier score and the H measure.

Main Results:

• The proposed weighted Brier scores effectively measure clinical utility.
• The decomposition highlights the importance of discrimination and calibration.
• Demonstrated practical application using prostate cancer data.

Conclusions:

• Weighted Brier scores offer a superior method for evaluating clinical utility in risk prediction.
• This approach enhances the assessment of prediction models for clinical decision-making.
• The method provides a robust framework for understanding model performance in real-world applications.