A nonparametric updating method to correct clinical prediction model drift.

Area of Science:

• Biostatistics
• Machine Learning
• Clinical Informatics

Background:

• Clinical prediction models degrade over time, necessitating updates to maintain performance.
• Existing updating methods may overfit or not account for sample size uncertainty.
• Parametric and nonparametric models require adaptable updating strategies.

Purpose of the Study:

• To develop a testing procedure for selecting optimal clinical prediction model updating methods.
• To minimize overfitting and incorporate uncertainty in updating sample sizes.
• To create a procedure applicable to both parametric and nonparametric models.

Main Methods:

• A procedure was developed to select updating methods for dichotomous outcome models.
• The procedure balances model simplicity against predictive accuracy.
• Method performance was evaluated using simulated population shifts and real-world clinical data.

Main Results:

• The test recommended appropriate updates based on simulated population shifts, including no update, recalibration, or refitting.
• Recommended updates generally achieved superior or similar calibration compared to more complex methods.
• A case study indicated the test's conservative nature, sometimes favoring simpler updates.

Conclusions:

• The developed test supports data-driven updating of clinical prediction models.
• It promotes the transportability and maintenance of models developed using biostatistical and machine learning approaches.
• This facilitates reliable applications relying on modern prediction tools.