Why most responder analyses are misleading

So-called responder analyses are commonly used in randomized controlled trials (RCT) for osteoarthritis and are typically based on observed change from baseline in self-reported pain. However, it is well known in the methodological literature that such responder analyses are misleading. We aimed to illustrate the size of the problem using simulation.

We generated individual pain trajectories based on real-life assumptions: normal distribution, mean pain 45 on visual analogue scale (VAS, range 0-100), within person standard deviation 12, between person standard deviation 25. Further, we generated plausible data from RCTs with true treatment effect on pain varying from 0 to 15 points on VAS and true proportion of responders 0% or 100%. We applied typical responder analysis to these generated trials.

With natural fluctuations of pain, the observed change in pain from baseline does not equal response to treatment. Even if a treatment is highly effective in reducing pain in all patients (100%) by 15 mm VAS, and no patient (0%) is responder to placebo, a typical responder analysis would suggest that 80% in the active treatment arm compared to 50% of persons in a placebo arm are responders, underestimating both the absolute and relative efficacy/effectiveness of the treatment and falsely implying heterogeneity in treatment effects.

Responder analysis based on change from baseline in VAS pain should be abandoned in analysis of parallel-group RCTs. Responder criteria based on change from baseline in other fluctuating outcomes, e.g. patients' self-reported symptoms, function and global assessment, should be scrutinized, as they likely share similar limitations."