Medical product safety surveillance: how many databases to use?

Large linked database networks, like the US Food and Drug Administration's Sentinel System, are being built for medical product surveillance. One use of these networks is for "near real-time" sequential database surveillance of prespecified medical product-adverse event pairs, which may result in a "safety signal" when a statistical excess risk is detected. Sequential database surveillance requires the investigator to manage surveillance in both information time (ie, how sample size accrues) and calendar time. Calendar time is important because people external to the surveillance population may be affected by the speed with which a safety signal is detected or ruled out. Optimal design and analysis aspects of sequential database surveillance are not well developed, but are gaining in importance as observational database networks grow. Using information time concepts, we show how to calculate sample sizes when performing sequential database surveillance, illustrating the relationships between statistical power, the time to detect a signal, and the maximum sample size for various true effect sizes. Then, using a vaccine example, we demonstrate a four-step planning process that allows investigators to translate information time into calendar time. Given the calendar time for surveillance, the process focuses on choosing observational database configurations consistent with the investigator's preferences for timeliness and statistical power. Although the planning process emphasizes sample size considerations, the influence of secondary database attributes such as delay times, measurement error, and cost are also discussed. Appropriate planning allows the most efficient use of public health dollars dedicated to medical product surveillance efforts.