Using cluster-based permutation tests to estimate MEG/EEG onsets: How bad is it?

Area of Science:

• Neuroscience
• Cognitive Science
• Biostatistics

Background:

• Magneto- and electroencephalography (EEG) research aims to localize neural effects in space and time.
• Cluster-based statistical methods are popular for correcting multiple comparisons in EEG data, but have limitations in precise temporal localization.

Purpose of the Study:

• To compare the performance of cluster-sum inferences with other statistical methods for localizing effects in EEG data.
• To evaluate strategies for reducing bias and variability in onset estimation.
• To introduce a novel method for generating reliable confidence intervals for effect onsets.

Main Methods:

• A simulation study comparing cluster-sum inferences with false discovery rate and familywise error rate controlling methods.
• Implementation of bias reduction strategies: group calibration, group comparison, and binary segmentation.
• Generation of onset hierarchical bootstrap confidence intervals.

Main Results:

• The cluster-sum method replicates previously reported positive bias in onset estimation.
• Despite bias, cluster-sum performs relatively well compared to pointwise p-value methods in terms of bias and variability.
• Binary segmentation outperformed mass-univariate methods in simulations for onset detection.
• Hierarchical bootstrap confidence intervals effectively integrate trial and participant variability.

Conclusions:

• Standard cluster-sum methods in EEG analysis may lead to biased onset estimations.
• Proposed strategies, particularly binary segmentation and hierarchical bootstrap confidence intervals, offer improved accuracy and reliability for localizing neural effects.
• These advancements are crucial for precise interpretation of temporal dynamics in EEG research.