Significance testing of many variables. Problems and solutions.

Area of Science:

• Pharmacoelectroencephalography (Pharmaco-EEG)
• Statistical analysis in neuroscience

Background:

• Pharmaco-EEG studies frequently analyze multiple variables, including repeated measurements over time.
• Multivariate analysis is often unsuitable for these complex datasets.
• Univariate testing of numerous variables inflates the probability of Type I errors (alpha-inflation).

Purpose of the Study:

• To address the challenge of alpha-inflation in pharmaco-EEG studies with multiple variables.
• To review and discuss various alpha-adjustment procedures for accurate statistical inference.

Main Methods:

• Discussion of established and proposed alpha-adjustment techniques.
• Examination of procedures designed to control the overall Type I error rate.
• Analysis of methods accommodating specific rejection criteria for null hypotheses.

Main Results:

• Alpha-inflation is a significant problem in pharmaco-EEG research due to multiple testing.
• Various alpha-adjustment methods can counteract the inflation of significance levels.
• Specific procedures are available for scenarios requiring a minimum rejection rate of null hypotheses.

Conclusions:

• Accurate statistical inference in pharmaco-EEG requires addressing alpha-inflation.
• Alpha-adjustment methods are crucial for maintaining valid significance levels.
• The choice of adjustment procedure depends on study design and desired statistical outcomes.