Updating methods improved the performance of a clinical prediction model in new patients

Ideally, clinical prediction models are generalizable to other patient groups. Unfortunately, they perform regularly worse when validated in new patients and are then often redeveloped. While the original prediction model usually has been developed on a large data set, redevelopment then often occurs on the smaller validation set. Recently, methods to update existing prediction models with the data of new patients have been proposed. We used an existing model that preoperatively predicts the risk of severe postoperative pain (SPP) to compare five updating methods.

The model was tested and updated with a set of 752 new patients (274 [36] with SPP). We studied the discrimination (ability to distinguish between patients with and without SPP) and calibration (agreement between the predicted risks and observed frequencies of SPP) of the five updated models in 283 other patients (100 [35%] with SPP).

Simple recalibration methods improved the calibration to a similar extent as revision methods that made more extensive adjustments to the original model. Discrimination could not be improved by any of the methods.

When the performance is poor in new patients, updating methods can be applied to adjust the model, rather than to develop a new model.