Comparing methods for mass univariate analyses of human EEG: Empirical data and simulations

Electroencephalography (EEG) is a widely used method for investigating human brain dynamics. However, EEG analyses are frequently conducted with limited a priori knowledge regarding locations or latencies of meaningful statistical effects. This makes it difficult for researchers to form regions of interest (ROIs), which are then analyzed using traditional statistical models such as analysis of variance. To address this, mass univariate analyses have become a valuable complement to ROI-based approaches. These methods attempt to correct for multiple comparisons while mitigating the risk of false positives and false negatives, thus enabling statistical inference in high-dimensional EEG data.

Here, we review and evaluate different approaches for delineating spatial and temporal effect boundaries in three datasets. Specifically, we focus on permutation-based approaches and their Bayesian alternatives to address within-subjects condition differences in i) steady-state evoked responses, ii) event-related potentials, and iii) time-frequency data.

Overall, simulation results showed variability in the number of datapoints indicating statistical condition differences.

Cluster-based permutation tests provide a relatively liberal approach to correct for multiple comparisons across domains, with high sensitivity for detecting large effects. In contrast, the permutation-based t_max procedure yields the most conservative method across datasets. Bayesian approaches are continuous in nature and thus strongly depend on the selection of thresholds for when support for a hypothesis is considered meaningful.

These findings provide insights into the strengths and limitations of current mass univariate approaches with the goal of supporting more informed decision-making in the analysis of EEG data.