Spike detection. I. Correlation and reliability of human experts

Area of Science:

• Neuroscience
• Biomedical Engineering
• Medical Statistics

Background:

• Epilepsy diagnosis relies on accurate spike detection in electroencephalography (EEG).
• Quantifying the reliability of human expert spike detection is crucial for validating diagnostic tools.
• Existing methods may not fully account for reader agreement on non-spike regions.

Purpose of the Study:

• To develop and apply novel statistical methods for quantifying inter-reader reliability in EEG spike detection.
• To compare the reliability of dichotomous versus continuous value models for spike perception.
• To establish a high-quality "gold standard" dataset for evaluating automated spike detection algorithms.

Main Methods:

• Five experienced electroencephalographers analyzed EEG trials from 40 epilepsy patients and 10 controls.
• A novel "detection correlation coefficient" was derived, extending Pearson correlation.
• Spike perception was modeled using both dichotomous and continuous values; reliability was assessed using measurement error theory.

Main Results:

• A total of 1952 spikes were detected with detailed attribute scoring.
• Study reliability was higher when using continuous values for spike perception compared to dichotomous values.
• The average inter-reader correlation was 0.79, with a corresponding reliability of 0.95, establishing a robust "panel score" database.

Conclusions:

• Continuous value modeling significantly enhances the reliability of EEG spike detection analysis.
• The derived "detection correlation coefficient" and "panel score" database provide a reliable benchmark for future research and algorithm development.
• This work offers a robust methodology for assessing human reader performance and creating gold standards in epilepsy diagnostics.