External validation of a model to predict future chronic opioid use among hospitalized patients

Previous research demonstrates an association between opioid prescribing at hospital discharge and future chronic opioid use. Various opioid guidelines and policies contributed to changes in opioid prescribing practices. How this affected hospitalized patients remains unknown.

Externally validate a prediction model to identify hospitalized patients at the highest risk for future chronic opioid therapy (COT).

Retrospective analysis of health record data from 2011 to 2022 using logistic regression.

Hospitalized adults with limited to no opioid use 1-year prior to hospitalization.

A statewide healthcare system.

Used variables associated with progression to COT in a derivation cohort from a different healthcare system to predict expected outcomes in the validation cohort.

The derivation cohort included 17,060 patients, of whom 9653 (56.6%) progressed to COT 1 year after discharge. Compared to the derivation cohort, in the validation cohort, patients who received indigent care (odds ratio [OR] = 0.40, 95% confidence interval [CI] =  0.27-0.59, p < .001) were least likely to progress to COT. Among variables assessed, opioid receipt at discharge was most strongly associated with progression to COT (OR = 3.74, 95% CI = 3.06-4.61, p < .001). The receiver operating characteristic curve for the validation set using coefficients from the derivation cohort performed slightly better than chance (AUC = 0.55).

Our results highlight the importance of externally validating a prediction model prior to use outside of the derivation population. Periodic updates to models are necessary as policy changes and clinical practice recommendations may affect model performance.