Signal detection based on time-to-onset: extending a new method from spontaneous reports to observational studies

A proof-of-concept study has previously highlighted the added value of a method using time-to-onset (TTO) for quantitative and non-parametric signal detection on spontaneous report data. The aim of this study was to assess the added value of this new TTO signal detection method adapted to observational studies.

For each adverse event collected during the conduct of an observational study of H1N1 pandemic influenza vaccine, the TTO distribution was tested against the 'follow-up distribution' from vaccination to 'lost to follow-up' by a Kolmogorov-Smirnov test. Events rejecting the null hypothesis of similar distribution were flagged as signals, and a safety physician evaluated their relevance for further medical assessment. We simulated ongoing surveillance by performing retrospective weekly signal detection based on TTO.

The TTO method detected 21, 15 and 4 signals within a 30-day period post-dose 1 with confidence levels set at 90%, 95% and 99%, respectively. Of these signals, 14 (67%), 10 (67%) and 2 (50%) were considered as relevant. Among the 14, six had not been identified by previous signal detection activities. When performed weekly, the Kolmogorov-Smirnov test detected 26 events as signals (alpha = 0.05). Three weeks after first participant first dose, one of the six new signals could theoretically have been detected.

This study provided evidence that the Kolmogorov-Smirnov method can screen all TTO distributions and objectively flag the unexpected, leading to earlier detection of signals, and thus potential safety issues.