Unresolved issues with noninferiority pragmatic trials: Results of a literature survey

Issues with specification of margins, adherence, and analytic population can potentially bias results toward the alternative in randomized noninferiority pragmatic trials. To investigate this potential for bias, we conducted a targeted search of the medical literature to examine how noninferiority pragmatic trials address these issues.

An Ovid MEDLINE database search was performed identifying publications in New England Journal of Medicine, Journal of the American Medical Association, Lancet, or British Medical Journal published between 2015 and 2021 that included the words "pragmatic" or "comparative effectiveness" and "noninferiority" or "non-inferiority." Our search identified 14 potential trials, 12 meeting our inclusion criteria (11 individually randomized, 1 cluster-randomized).

Eleven trials had results that met the criteria established for noninferiority. Noninferiority margins were prespecified for all trials; all but two trials provided justification of the margin. Most trials did some monitoring of treatment adherence. All trials conducted intent-to-treat or modified intent-to-treat analyses along with per-protocol analyses and these analyses reached similar conclusions. Only two trials included all randomized participants in the primary analysis, one used multiple imputation for missing data. The percentage excluded from primary analyses ranged from ∼2% to 30%. Reasons for exclusion included randomization in error, nonadherence, not receiving assigned treatment, death, withdrawal, lost to follow-up, and incomplete data.

Specification of margins, adherence, and analytic population require careful consideration to prevent bias toward the alternative in noninferiority pragmatic trials. Although separate guidance has been developed for noninferiority and pragmatic trials, it is not compatible with conducting a noninferiority pragmatic trial. Hence, these trials should probably not be done in their current format without developing new guidelines.