Partition testing for real-world evidence studies

Traditional approaches to hypothesis testing in comparative post-approval safety and effectiveness studies of medical products are often inadequate because of a limited scope of possible inferences (e.g., superiority or inferiority). Often there is interest in simultaneously testing for superiority, equivalence, inferiority, non-inferiority, and non-superiority, which can be achieved using a partition testing framework. Partition testing only requires selection of an equivalence margin and calculation of a two-sided Wald confidence interval. In addition to permitting a broader range of inferences, the strengths of the approach include: mitigating publication bias, avoiding use of a clinically irrelevant nil hypothesis, and more transparent and impartial appraisal of the clinical importance of a study's findings by pre-specifying an equivalence margin. However, a challenge in implementing the approach can be the process for identifying an equivalence margin. The methodology is illustrated using a published study of the safety of Ondansetron for the off-label treatment of nausea and vomiting during pregnancy. Applying the method to the study results would have led to a conclusion that women exposed to Ondansetron in comparison to those that are not, are equivalent with respect to risk of cardiac malformations and oral clefts. These conclusions are more in line with the magnitude of the observed effects than the conclusions resulting from a traditional inferiority/superiority testing conducted by the study authors.