Changes in estimands and prevalence of discrepancies in conclusions under the two-trial paradigm: a systematic review

The United States Food and Drug Administration (FDA) has recommended that two adequate and well-controlled trials are conducted to establish efficacy. Although replication strengthens evidentiary robustness, conducting two trials may raise interpretative issues in case of discrepant findings, or if knowledge gained from the first trial influences the conduct of the second. The objectives of this study are to describe how the strict "two-trial rule" (ie, duplicated trials) is applied to phase 3 trials, especially regarding outcome modifications, and how discrepancies between trials are handled from a regulatory perspective.

We extracted all phase 3 trials from ClinicalTrials.gov between January 1, 2010, and January 7, 2024. We automatically retrieved duplicated trials by searching identical sponsor, disease, intervention, and study design. All pairs were checked by two investigators, and discrepancies were discussed among authors. We extracted all outcome modifications, study results, and subsequently searched how discrepant conclusions and outcome modifications were considered by the FDA.

Among 9925 eligible trials, there were 498 duplicated studies (5%; 246 pairs or triplets), mainly in dermatology (n = 96, 19%) and ophthalmology (n = 58, 12%). Among them, 86 trials did not report their primary end point (n = 56, 11%) or did but not properly (n = 30, 6%). Changes in primary outcome or in the type 1 error control method during the trial were observed for 29 pairs (12%), and mostly concerned all trials within a pair (n = 17, 7%). For nine pairs (4%), changes were performed in one trial after unblinding of the first trial. Study results were available for 206 pairs (70%), among which discrepancies in statistical conclusion were found in 35 (17%). FDA authorization was granted for 14 drugs despite these discrepancies, among which an additional randomized controlled trial was required for eight cases.

Although the two-trial paradigm has been the default rule for years, duplicated trials only represent about 5% of phase 3 trials between 2010 and 2024. Changes in outcomes that may seriously compromise the robustness of the results were observed in less than 5% of situations. However, there is a lack of consistency in marketing decisions in case of discrepant results.