Quality of patient-reported outcome measures and treatment effects: a review of orthopaedic randomized trials

This study assessed how poor-quality patients reported outcome measures (PROMS) influence treatment effect estimates in randomized patients with anterior cruciate ligament (ACL) injury.

We chose a representative sample of studies by searching top journals of orthopaedic surgery for randomized trials in patients after anterior cruciate ligament injury over the past 5-years. Studies were eligible if they reported at least one of the PROMs assessed in a previous review of their measurement properties. The risk of bias was assessed, and standardized treatment effects were calculated between groups for each PROM and time point. We used mixed-effects regression to predict treatment effect using psychometric summary score, risk of bias score, follow-up duration, sample size, proportion of female patients, and funding source as predictors, accounting for the random effect of the study.

Out of the 4038 studies screened, 38 RCTs that used 9 validated PROMs and reported 139 PROM scores were included. The mean risk of bias score was 7.1 (standard deviation [SD]: 1), and the mean psychometric score was 1.4 (SD = 2.5). While psychometric quality was not significantly associated with treatment effect variance in lower-quality PROMs, the study found that longer follow-up duration was associated with a less likely treatment effect favouring the experimental group.

Poor quality patient-reported outcomes may demonstrate greater variance of treatment effects in randomized trials. More psychometric evidence, particularly of responsiveness, is needed to assess the effect of individual measurement properties on treatment effects. These findings should be considered hypothesis generating only and be followed up in additional related research.