Spike detection: Inter-reader agreement and a statistical Turing test on a large data set.

Area of Science:

• Neuroscience
• Medical Technology
• Signal Processing

Background:

• Accurate spike detection in electroencephalograms (EEGs) is crucial for diagnosing neurological disorders.
• Human interpretation of EEG spikes can be subjective, leading to inter-rater variability.

Purpose of the Study:

• To compare the spike detection performance of skilled humans with computer algorithms.
• To establish a benchmark for algorithm performance against human expertise.

Main Methods:

• 40 prolonged EEGs were analyzed, with 35 containing reported spikes.
• Spikes and sharp waves were marked by three human experts and three computer algorithms.
• Pairwise sensitivity and false positive rates were calculated for human-human and algorithm-human comparisons.
• A statistical Turing test methodology was employed to compare performance differences.

Main Results:

• Humans exhibited fair agreement in spike marking, with mean pairwise sensitivities ranging from 40.0% to 51.5%.
• The Persyst 13 (P13) algorithm achieved a sensitivity of 43.9% and a false positive rate of 1.65/min, comparable to human performance.
• Statistical analysis confirmed that P13 met noninferiority criteria compared to human performance.

Conclusions:

• Skilled humans show limited agreement in marking EEG spikes.
• The P13 algorithm achieved statistically noninferior spike detection performance compared to human experts.
• This study introduces a novel methodology for evaluating algorithms against human performance standards when a gold standard is absent.