Assessing calibration of multinomial risk prediction models

Area of Science:

• Biostatistics
• Medical Informatics
• Epidemiology

Background:

• Model calibration is crucial for evaluating risk prediction accuracy.
• Existing calibration tools are limited to dichotomous outcomes.
• Extending calibration methods to polytomous outcomes is needed.

Purpose of the Study:

• To develop and validate calibration assessment tools for risk prediction models with polytomous outcomes.
• To adapt existing calibration techniques like logistic recalibration for multinomial logistic regression (MLR).

Main Methods:

• Proposed a multinomial logistic recalibration framework using MLR.
• Introduced a non-parametric alternative employing vector splines.
• Developed parametric and non-parametric multinomial calibration plots.
• Enabled estimation and testing of calibration intercepts and slopes.

Main Results:

• Demonstrated the applicability of the proposed framework in two case studies (ovarian tumor malignancy, testicular cancer residual mass).
• External validation confirmed the utility of the developed methods.
• Polytomous calibration plots provide informative visual summaries of model calibration.

Conclusions:

• Calibration assessment tools can be effectively extended to polytomous outcomes.
• The proposed multinomial calibration plots offer valuable insights into prediction model performance.
• These advancements improve the evaluation of complex risk prediction models.