Rating by detection: an artifact detection protocol for rating EEG quality with average event duration

Objective. Quantitative evaluation protocols are critical for the development of algorithms that remove artifacts from real electroencephalography (EEG) optimally. However, visually inspecting the real EEG to select the top-performing artifact removal pipeline is infeasible while hand-crafted EEG data allow assessing artifact removal configurations only in a simulated environment. This study proposes a novel, principled approach for quantitatively evaluating algorithmically corrected EEG without access to ground truth in real-world conditions.Approach. Our offline evaluation protocol uses a detector to score the presence of artifacts. It computes their average duration, which measures the recovered EEG's deviation from the modeled background activity with a single score. As we expect the detector to make generalization errors, we employ a generic and configurable Wiener-based artifact removal method to validate the reliability of our detection protocol.Main results. Quantitative experiments extensively compare many Wiener filters and show their consistent rankings agree with their theoretical assumptions and expectations.Significance. The rating-by-detection protocol with the average event duration measure should be of value for EEG practitioners and developers. After removing artifacts from real EEG, the protocol experimentally shows that reliable comparisons between many artifact filtering configurations are possible despite the missing ground-truth neural signals.