Comparing Methods for Mass Univariate Analyses of Human EEG: Empirical Data and Simulations

Electroencephalography (EEG) is a widely used method for investigating human brain dynamics. However, EEG analyses are frequently conducted with limited a priori knowledge regarding locations or latencies of meaningful statistical effects. This makes it difficult for researchers to form regions of interest (ROIs), which are then analyzed using traditional statistical models such as analysis of variance. In addition, exploratory studies, or studies interested in determining the exact temporal and spatial extent of a predicted effect may aim to examine many sensor locations and time points, often jointly. To address this, mass univariate analyses have become a valuable complement to ROI-based approaches. These methods attempt to correct for multiple comparisons while mitigating the risk of false positives and false negatives, thus enabling statistical inference in high-dimensional EEG data. Here, we review and evaluate different approaches for delineating spatial and temporal effect boundaries in three different datasets, focusing on within-subjects comparisons. Specifically, we focus on permutation-based approaches and their Bayesian alternatives to address condition differences in i) steady-state evoked responses, ii) event-related potentials, and iii) time-frequency data. Overall, simulation results indicate that cluster-based permutation tests provide a relatively liberal approach to correct for multiple comparisons across domains, with high sensitivity for detecting large effects. In contrast, the permutation-based t _max procedure yields the most conservative method across datasets. Bayesian approaches inherently are continuous in nature and thus strongly depend on the selection of thresholds for when support for a hypothesis is considered meaningful.