The risk of severe postoperative pain: modification and validation of a clinical prediction rule

Recently, a prediction rule was developed to preoperatively predict the risk of severe pain in the first postoperative hour in surgical inpatients. We aimed to modify the rule to enhance its use in both surgical inpatients and outpatients (ambulatory patients). Subsequently, we prospectively tested the modified rule in patients who underwent surgery later in time and in another hospital (external validation).

The rule was originally developed from the data of 1395 adult inpatients. We modified the rule with the data of 549 outpatients who underwent surgery between 1997 and 1999 in the same center (Academic Medical Center Amsterdam, The Netherlands). Furthermore, we tested the performance of the modified rule in 1035 in- and outpatients who underwent surgery in 2004, in the University Medical Center Utrecht, The Netherlands (external validation). Performance was quantified by the rule's calibration (agreement between observed frequencies and predicted risks) and discrimination (ability to distinguish between patients at high and low risk).

Modification of the original rule to enhance prediction in outpatients included reclassification of the predictor "type of surgery," addition of the predictor "surgical setting" (ambulatory surgery: yes/no) and addition of interaction terms between surgical setting and the other predictors. One-third of the patients in the Utrecht cohort reported severe postoperative pain (36%), compared to 62% of the patients in the Amsterdam cohort. The distribution of most predictors was similar in the two cohorts, although the patients in the Utrecht cohort were slightly older, more often underwent ambulatory surgery and had large expected incision sizes less often than patients in the Amsterdam cohort. The modified prediction rule showed good calibration, when an adjusted intercept was used for the lower incidence in the Utrecht cohort. The discrimination was reasonable (area under the Receiver Operating Characteristic curve 0.65 [95% confidence interval 0.57-0.73]).

A previously developed prediction rule to predict severe postoperative pain was modified to allow use in both inpatients and outpatients. By validating the rule in patients who underwent surgery several years later in another hospital, it was shown that the rule could be generalized in time and place. We demonstrated that, instead of deriving new prediction rules for new populations, a simple adjustment may be enough to recalibrate prediction rules for new populations. This is in line with the perception that external validation and updating of prediction rules is a continuing and multistage process.