Explanation of Two Anomalous Results in Statistical Mediation Analysis

Area of Science:

• Statistics
• Psychometrics
• Social Sciences Research Methods

Background:

• Mediation models are crucial for understanding indirect effects in various scientific fields.
• Bias-corrected bootstrap methods are popular for testing mediation due to perceived higher statistical power.
• Previous research noted anomalous results with bias-corrected bootstrap tests, including inflated Type I errors and power limitations.

Purpose of the Study:

• To investigate the anomalous findings in bias-corrected and accelerated bias-corrected bootstrap tests for mediation models.
• To examine the conditions under which Type I error rates become elevated.
• To understand the factors contributing to statistical power asymptotes and declines in mediation analysis.

Main Methods:

• Conducted two computer simulations to rigorously test mediation model assumptions.
• Simulation 1 focused on Type I error rates under varying path sizes and sample sizes.
• Simulation 2 examined statistical power as a function of effect sizes for paths 'a' (X to M) and 'b' (M to Y).

Main Results:

• Elevated Type I error rates for bias-corrected bootstrap tests occur with small sample sizes and medium to large effect sizes in individual paths.
• Statistical power stagnates or declines when the 'a' path effect size increases, particularly when the 'b' path effect size is small.
• Empirical examples using data from a youth steroid prevention program illustrate these Type I error and power issues.

Conclusions:

• The bias-corrected bootstrap test can yield unreliable results (inflated Type I errors) under specific sample size and effect size conditions.
• Researchers must be cautious about statistical power limitations in mediation analysis, especially when the indirect effect is small.
• Findings highlight the need for careful consideration of test selection and interpretation in mediation analysis to ensure valid conclusions.