Statistically comparing EEG/MEG waveforms through successive significant univariate tests: how bad can it be?
Related Experiment Video
Updated: Apr 23, 2026
Utilizing Electroencephalography Measurements for Comparison of Task-Specific Neural Efficiencies: Spatial Intelligence Tasks
Published on: August 9, 2016
Vitória Piai1, Kristoffer Dahlslätt, Eric Maris
1Department of Psychology and the Helen Wills Neuroscience Institute, University of California at Berkeley, Berkeley, California, USA; Radboud University Nijmegen, Donders Institute for Brain, Cognition and Behaviour, Nijmegen, The Netherlands.
Abstract:
When making statistical comparisons, the temporal dimension of the EEG signal introduces problems. Guthrie and Buchwald (1991) proposed a formally correct statistical approach that deals with these problems: comparing waveforms by counting the number of successive significant univariate tests and then contrasting this number to a well-chosen critical value. However, in the literature, this method is often used inappropriately. Using real EEG data and Monte Carlo simulations, we examined the problems associated with the incorrect use of this approach under circumstances often encountered in the literature. Our results show inflated false-positive or false-negative rates depending on parameters of the data, including filtering. Our findings suggest that most applications of this method result in an inappropriate familywise error rate control. Solutions and alternative methods are discussed.
