The Fallacy of Patient-Reported Outcome Measures: Why They Often Fail to Detect Clinically Important Differences.

Area of Science:

• Orthopaedic Surgery
• Clinical Trial Design
• Biostatistics

Background:

• Patient-reported outcome measures (PROMs) are increasingly used in orthopaedic research.
• However, PROMs often fail to detect clinically meaningful differences in clinical trials.
• This study compares the ability of binary versus continuous PROMs to detect significant differences.

Purpose of the Study:

• To assess if orthopaedic studies are more likely to detect clinically important differences using a binary outcome (e.g., nonunion surgery) or a continuous patient-reported outcome.
• To compare the statistical power of binary versus continuous outcomes in detecting treatment effects.

Main Methods:

• A hypothetical clinical trial comparing two treatments for tibial shaft fractures was simulated.
• Assumed a 10% nonunion rate in the treatment group and 20% in the control group.
• Simulated 10,000 trials with 400 patients, analyzing nonunion surgery rates and Patient-Reported Outcomes Measurement Information System (PROMIS) physical function scores.

Main Results:

• The binary outcome of nonunion surgery was statistically significant in 80.9% of simulated trials.
• The continuous PROMIS physical function score was statistically significant in only 4.2% of trials.
• A significant difference in detecting outcomes was observed (p < 0.001).

Conclusions:

• Studies using PROMs may fail to detect statistically significant differences when the clinically important outcome occurs in a minority of patients.
• Orthopaedic surgeons and researchers should consider this limitation when designing and evaluating studies.
• Awareness of PROM limitations is crucial for accurate treatment comparison.