Improving patient-reported outcome measure scoring in shoulder trials : comparing classical test theory, item...

Patient-reported outcome measures (PROMs) underpin orthopaedic trials by capturing pain and function from the patient's perspective. Traditionally, PROMs such as the Oxford Shoulder Score (OSS) are scored using classical test theory (CTT), which assumes equal item contribution and uniform measurement error. Item response theory (IRT) and computerized adaptive testing (CAT) offer modern alternatives that may improve precision and reduce respondent burden, but their impact on trial outcomes is unclear.

We reanalyzed patient-level OSS data from two major UK randomized controlled trials, UK FROST (n = 503) and PROFHER (n = 250). We used three scoring approaches: CTT sum scores, IRT-based expected a posteriori (EAP) scores, and a simulated CAT. Original statistical analysis plans were replicated to compare treatment effect estimates, CIs, and p-values across methods. CAT simulations assessed potential reductions in questionnaire length.

Across both trials, treatment effect estimates and statistical significance were highly consistent between CTT, IRT, and CAT. Differences in magnitude were minimal, and overall trial conclusions were unchanged. Simulated CAT reduced median questionnaire length to four to five items at early follow-up, increasing to eight to nine items at later timepoints.

In these large orthopaedic trials, modern psychometric methods produced results consistent with traditional scoring, but demonstrated potential efficiency gains. IRT and CAT may facilitate lower-burden, patient-centred outcome measurement and support innovative trial designs, without compromising the interpretation of treatment effects.