Performance assessment for EEG-based neonatal seizure detectors

Area of Science:

• Biomedical Engineering
• Clinical Neurophysiology
• Machine Learning in Healthcare

Background:

• Neonatal seizures are a critical concern in neonatal intensive care units (NICUs).
• Accurate and reliable detection of neonatal seizures using electroencephalography (EEG) is challenging.
• Existing performance assessment methods for EEG-based seizure detection systems may be insufficient.

Purpose of the Study:

• To establish a robust framework for measuring the performance of neonatal seizure detection systems.
• To present and evaluate a multi-channel, patient-independent neonatal seizure detection system utilizing a Support Vector Machine (SVM) classifier.
• To introduce a novel metric for assessing the average duration of false detections.

Main Methods:

• Development of a comprehensive performance assessment framework including metrics, experimental setups, and testing protocols.
• Evaluation of an SVM-based neonatal seizure detection system within the established framework.
• Calculation of epoch-based and event-based performance metrics, including a new false detection duration metric.
• Investigation of two post-processing steps to enhance temporal precision and robustness.
• Validation of the system on a large clinical dataset (267 hours).

Main Results:

• The proposed framework highlights the necessity of a complete metric set and specific testing protocols for objective performance assessment and comparison.
• The developed SVM-based system achieved a record ROC area of 96.3%, with ~90% sensitivity and specificity at the equal error rate.
• The system demonstrated an average detection rate of ~89% with one false detection per hour, averaging 2.7 minutes in duration.

Conclusions:

• Accurate performance assessment of EEG-based neonatal seizure detectors requires reporting multiple metrics and adhering to a specific testing protocol.
• Relying solely on event-based metrics can be misleading and may not fully represent system performance.
• The evaluated SVM system offers significant potential to assist clinical staff in interpreting EEG data in NICUs.