The statistics of a practical seizure warning system

Area of Science:

• Neurology
• Biomedical Engineering
• Statistics

Background:

• Statistical methods for evaluating seizure prediction algorithms are controversial, hindering clinical application.
• Current methods for comparing algorithm performance to chance are debated.
• Seizure prediction algorithms must demonstrably outperform random chance.

Purpose of the Study:

• To derive a statistical framework for comparing seizure prediction algorithms to chance performance.
• To propose a new test metric for evaluating seizure advisory systems.
• To demonstrate the utility of the proposed methods using real patient data.

Main Methods:

• Derived a statistical framework using the expected performance of a chance predictor as a control in a hypothesis test.
• Verified chance prediction performance using Monte Carlo simulations with variable duration seizure warnings.
• Proposed a new test metric: difference between algorithm and chance sensitivities under a time-in-warning constraint.

Main Results:

• Developed a statistically sound method for evaluating seizure prediction algorithms against chance.
• Demonstrated the utility of the new metric using spectral power-based seizure prediction in four epilepsy surgery patients.
• The proposed methods are broadly applicable to various scoring rules.

Conclusions:

• The derived statistical framework provides a reliable method for evaluating seizure prediction algorithms.
• The new test metric offers a practical advance for assessing seizure advisory systems.
• This work addresses a primary barrier to the clinical realization of seizure prediction technology.