Robust Alpha Spending for Unstable Data

Sequential hypothesis testing has been used in randomized clinical trials and in monitoring adverse events after vaccination. Typically, the presumption is that all data that have been previously used in the sequential analysis remain the same, and only new data are added. We call this the "anchoring" assumption. In prelicensing randomized trials, enrollment goals are set based on these sequential hypothesis tests so that a trial may stop if a drug proves either too hazardous or, conversely, so beneficial that it would be unethical to deny the treatment to others. However, in monitoring adverse events after vaccination, data are mostly "secondary use," or real-world data. That is, these data are not gathered for research, but are clinical data that have been repurposed for observational vaccine safety surveillance. As such, the counts of exposed persons and adverse events may be subject to revisions over time. These data changes directly challenge the anchoring assumption upon which most sequential hypothesis testing depends. If these data changes are discovered prior to subsequent hypothesis tests, it is unclear how to account for these changes in subsequent tests. We introduce a solution: "robust alpha spending" to protect overall type I error probability, statistical power, and surveillance time from low-level data instability present in secondary use data sources used for adverse event monitoring.