A Two-Stage Decision Making Approach for Safety Studies

For safety studies, two types of hypothesis testing are often considered: detecting a safety signal and ruling out a safety concern. Under the detecting framework, statistical non-significance is often confused with the conclusion that there is no safety concern. Such a conclusion, in the presence of low study power or large variability, is problematic. To overcome the interpretation issue with non-significant results from a detecting hypothesis, we propose a Two-Stage Decision-Making (TSDM) approach for safety studies. It is basically a ruling-out design allowing an interim analysis that applies both detecting and ruling-out criteria at the interim and final stages with a pre-specified alpha spending function. The proposed TSDM approach incorporates both detecting a safety signal and ruling out safety concerns into a single study design to increase the probability of making a definite decision. It is based on the ruling-out framework that utilizes both directions of the confidence interval to make a decision for ruling out unacceptable risk or detecting safety signal at each analysis stage. We assess the proposed TSDM approach by investigating properties such as operational type I error rate, overall study power based on analytical approximations, overall probability of making a decision, and required sample sizes. We conduct Monte Carlo simulations to evaluate such properties regarding various outcome types of confidence intervals and summarize the statistical interpretations and the implications on study design.