A Rational Approach to Understanding and Evaluating Responsive Neurostimulation

Closed-loop brain stimulation is increasingly used in level 4 epilepsy centers without an understanding of how the device behaves on a daily basis. This lack of insight is a barrier to improving closed-loop therapy and ultimately understanding why some patients never achieve seizure reduction. We aimed to quantify the accuracy of closed-loop seizure detection and stimulation on the RNS device through extrapolating information derived from manually reviewed ECoG recordings and comprehensive device logging information. RNS System event logging data were obtained, reviewed, and analyzed using a custom-built software package. A weighted-means methodology was developed to adjust for bias and incompleteness in event logs and evaluated using Bland-Altman plots and Wilcoxon signed-rank tests to compare adjusted and non-weighted (standard method) results. Twelve patients implanted for a mean of 21.5 (interquartile range 13.5-31) months were reviewed. The mean seizure frequency reduction post-RNS implantation was 40.1% (interquartile range 0-96.2%). Three primary levels of event logging granularity were identified (ECoG recordings: 3.0% complete (interquartile range 0.3-1.8%); Event Lists: 72.9% complete (interquartile range 44.7-99.8%); Activity Logs: 100% complete; completeness measured with respect to Activity Logs). Bland-Altman interpretation confirmed non-equivalence with unpredictable differences in both magnitude and direction. Wilcoxon signed rank tests demonstrated significant (p < 10^-6) differences in accuracy, sensitivity, and specificity at >5% absolute mean difference for extrapolated versus standard results. Device behavior logged by the RNS System should be used in conjunction with careful review of stored ECoG data to extrapolate metrics for detector performance and stimulation.