Sample Size Critically Shapes the Reliability of EEG Case-Control Findings in Psychiatry.

Area of Science:

• Neuroscience
• Psychiatry
• Biomedical Engineering

Background:

• Electroencephalography (EEG) studies in psychiatry often yield inconsistent findings, particularly in case-control designs.
• The impact of small sample sizes on the reliability of EEG group comparisons remains under-quantified.
• This variability hinders biomarker discovery in psychiatric research.

Purpose of the Study:

• To systematically quantify the effect of sample size on the reliability and reproducibility of EEG group comparisons.
• To assess the influence of sample size on effect sizes, statistical power, and false positive rates in psychiatric EEG research.
• To evaluate the utility of conventional case-control EEG approaches for biomarker discovery.

Main Methods:

• Utilized a large, multisite resting-state EEG dataset (N=2,874) of participants aged 5-18 years, including clinical and control groups.
• Extracted comprehensive spectral, temporal, and complexity EEG features.
• Performed extensive random subsampling for repeated case-control comparisons across various sample sizes.

Main Results:

• Small sample sizes resulted in unstable EEG findings with inflated and highly variable effect sizes.
• Larger sample sizes yielded consistent, reproducible results with small but robust effects.
• Statistical power increased significantly with sample size, while false positive rates remained stable.

Conclusions:

• Sample size critically influences the outcomes and reliability of psychiatric EEG studies.
• Conventional case-control EEG designs with small samples are inadequate for robust biomarker discovery.
• Larger sample sizes are essential for advancing psychiatric EEG research and identifying reliable biomarkers.