Are seizure forecasts and cycles better than chance? What chance?

There is a growing synergy between the lines of research on cycles in epilepsy and seizure forecasting. It has been conjectured, for instance, that incorporating information about significant seizure cycles into forecasting algorithms can lead to a better-than-chance forecasting performance. However, significance and better-than-chance are each typically evaluated against only a single null hypothesis, for example, that forecasts are generated by a Poisson process. We here argue that this should be considered only a first step. Our objective is to demonstrate the importance of testing complementary null hypotheses that represent alternative chance models.

To ensure controlled conditions, we use synthetic data generated from simple mathematical models. Samples drawn from gamma distributions are used to generate sequences of random seizure times and random forecasts. We then determine the strength of cycles as a function of the cycle duration and calculate the sensitivity and fraction of time under alarm obtained for the random forecasting algorithm. In both analyses, we apply numerical, surrogate-based null-hypothesis testing methods. In the latter case, this includes a straightforward approach to correcting for multiple testing on nonindependent data.

Counterintuitively, the random seizure-time sequences contain multiple prominent cycles, which are judged highly significant by the Rayleigh test. Moreover, randomly forecasting random seizure times results in a sensitivity of 79% at a fraction of time under alarm of only 42%, clearly outperforming a Poisson-like predictor. In both cases, however, the flexibility and versatility of surrogate-based null-hypothesis tests allow us to successfully reveal that all results can be explained by chance models.

Before reaching conclusions on real cycles in epilepsy, the forecastability of seizures, and genuine capacity of forecasting algorithms, it is essential to test and reject several complementary null hypotheses. Many conclusions might not withstand such rigorous tests, allowing the community to focus on those that do.