Heterogeneity of treatment effects: implications for guidelines, payment, and quality assessment

Randomized controlled trial results are needed for developing guidelines, payment rules, and quality-of-care measures; however, 2 phenomena reduce the usefulness of randomized controlled trial findings. First, these studies now enroll patients with less severe disease, who are less likely to benefit from a drug or treatment. Second, patients are living longer but, as a result, have more chronic diseases. Although randomized controlled trials often exclude these older patients, trial findings continue to be generalized to them. Together, these phenomena impose challenges to the usefulness of the results of randomized controlled trials for clinical and policy applications. The convergence of these phenomena makes the current research paradigm underlying evidence-based medicine, guideline development and quality assessment fundamentally flawed in 2 ways. First, the "evidence" includes patients who may have a minimal benefit from the treatment being tested. This could reduce the power for the trial and yield negative or null results, leading to undertreatment of a group of patients with potential for a greater-than-observed benefit. Second, attempts to generalize the results from positive trials to patients who have been excluded from those trials may result in the overtreatment of those who could not benefit (e.g., because they will die from other causes before the benefit of treatment would occur) and therefore represents a parallel hazard. In this article, we describe sources of heterogeneity of treatment effects (HTE) within trials, which can compromise the interpretation and generalizability of results. We also examine strategies for understanding and managing HTE in practice, to increase the usefulness of trial results.