Related Experiment Video
Updated: Dec 29, 2025
Problem-Solving Before Instruction PS-I: A Protocol for Assessment and Intervention in Students with Different Abilities
Published on: September 11, 2021
Permutation-based methods for mediation analysis in studies with small sample sizes
Miranda E Kroehl1, Sharon Lutz2, Brandie D Wagner1
1Department of Biostatistics and Informatics, Colorado School of Public Health, University of Colorado Anschutz Medical Campus, Aurora, CO, USA.
Background:
Mediation analysis can be used to evaluate the effect of an exposure on an outcome acting through an intermediate variable or mediator. For studies with small sample sizes, permutation testing may be useful in evaluating the indirect effect (i.e., the effect of exposure on the outcome through the mediator) while maintaining the appropriate type I error rate. For mediation analysis in studies with small sample sizes, existing permutation testing methods permute the residuals under the full or alternative model, but have not been evaluated under situations where covariates are included. In this article, we consider and evaluate two additional permutation approaches for testing the indirect effect in mediation analysis based on permutating the residuals under the reduced or null model which allows for the inclusion of covariates.
Methods:
Simulation studies were used to empirically evaluate the behavior of these two additional approaches: (1) the permutation test of the Indirect Effect under Reduced Models (IERM) and (2) the Permutation Supremum test under Reduced Models (PSRM). The performance of these methods was compared to the standard permutation approach for mediation analysis, the permutation test of the Indirect Effect under Full Models (IEFM). We evaluated the type 1 error rates and power of these methods in the presence of covariates since mediation analysis assumes no unmeasured confounders of the exposure-mediator-outcome relationships.
Results:
The proposed PSRM approach maintained type I error rates below nominal levels under all conditions, while the proposed IERM approach exhibited grossly inflated type I rates in many conditions and the standard IEFM exhibited inflated type I error rates under a small number of conditions. Power did not differ substantially between the proposed PSRM approach and the standard IEFM approach.
Conclusions:
The proposed PSRM approach is recommended over the existing IEFM approach for mediation analysis in studies with small sample sizes.
Insights
For small sample mediation analysis, the Permutation Supremum test under Reduced Models (PSRM) maintains accurate error rates, unlike other permutation tests. This method is recommended for reliable indirect effect evaluation in limited data scenarios.
Area of Science:
- Statistics
- Biostatistics
- Epidemiology
Background:
- Mediation analysis assesses exposure effects on outcomes via intermediate variables (mediators).
- Permutation testing is valuable for evaluating indirect effects in small sample sizes, maintaining type I error rates.
- Existing permutation methods for small sample mediation analysis have limitations when covariates are included.
Purpose of the Study:
- To evaluate two novel permutation approaches for testing indirect effects in mediation analysis with covariates.
- To compare these new methods against the standard permutation approach for small sample sizes.
Main Methods:
- Simulation studies were conducted to assess type I error rates and statistical power.
- Two new methods, Indirect Effect under Reduced Models (IERM) and Permutation Supremum test under Reduced Models (PSRM), were evaluated.
- The performance was compared to the Indirect Effect under Full Models (IEFM) approach.
Main Results:
- The PSRM approach effectively controlled type I error rates across all simulated conditions.
- The IERM approach demonstrated inflated type I error rates in many scenarios.
- The standard IEFM approach showed inflated type I error rates under specific conditions, while power was comparable between PSRM and IEFM.
Conclusions:
- The PSRM approach is recommended for mediation analysis in small sample studies.
- PSRM offers a reliable method for assessing indirect effects when covariates are present.
- The findings suggest PSRM is a superior alternative to existing methods like IEFM for small sample mediation analysis.
More Related Videos
06:26Meta-analysis of Voxel-Based Neuroimaging Studies using Seed-based d Mapping with Permutation of Subject Images SDM-PSI
Published on: November 27, 2019
08:36Author Spotlight: Evaluating the Adjuvant Efficacy and Safety of Angong Niuhuang Pill in Viral Encephalitis Treatment
Published on: April 19, 2024
Related Concept Videos
01:15One-Way ANOVA: Unequal Sample Sizes
01:21Friedman Two-way Analysis of Variance by Ranks
01:15One-Way ANOVA: Equal Sample Sizes
Different sample means can result in different values for the variance estimate: variance between samples. This is because the variance between samples is calculated as the product of the sample size and the variance between the...
01:13Randomized Experiments
Simple randomization
Simple...
01:23McNemar's Test
01:23Mechanistic Models: Compartment Models in Individual and Population Analysis