Substantial evidence of effect

Pressures for rapid drug development, especially for treatments that may affect public health significantly, drive a need to reconsider what is necessary to establish the "substantial evidence" of efficacy and safety required for regulatory approval. The concept of substantial evidence of effect can be stated fairly simply in principle, but its application to the evaluation of findings from real clinical drug development programs can be quite complicated and depend on circumstances relating to the population studied and the condition being treated. This paper discusses a number of considerations that arise in attempting to address this issue. These include confirmation of efficacy/safety as opposed to replication of results, the use of surrogates for the clinical outcome, ethical considerations, the use of trials aimed at demonstrating equivalence or noninferiority instead of superiority of a new drug, and the balance between benefits and risks. Recent developments such as the International Conference on Harmonization (ICH) guidelines dealing with statistical principles and choice of control group, and Food and Drug Administration (FDA) guidances interpreting the 1998 Modernization Act identify useful alternative definitions of substantial evidence and also identify when a single study can be enough to demonstrate a clinically important effect. The primary difficulty with the use of noninferiority trials is the need to demonstrate assay sensitivity or validity, i.e., that the trial would have demonstrated the superiority of the active control treatment to placebo had the trial included a placebo group. Failure to demonstrate a significant difference between the presumed active treatments and an inactive control precludes a definitive conclusion of assay validity, but external evidence, e.g., from trials demonstrating the activity of the active control, may yet justify concluding that the trial was assay valid. Issues of interpretation of important unexpected findings are illustrated in the recent experience with the alpha- and beta-blocker carvedilol, where there was a significant reduction in mortality among patients on carvedilol relative to placebo, but a treatment effect could not be demonstrated for the primary outcome. Experience with trials of thrombolytic agents illustrates alternative strategies for demonstrating substantial evidence of effect. Finally, meta-analyses can contribute to the substantial evidence by synthesizing the findings from separate trials to provide a perspective on drug performance in a heterogeneous population of patients that no single trial could provide.