nmax and the quest to restore caution, integrity, and practicality to the sample size planning process

Area of Science:

• Psychological Science
• Statistical Modeling

Background:

• Research replicability is a growing concern, necessitating robust statistical and inferential integrity in studies.
• Funding agencies require a priori power analyses to justify sample sizes for sound research investment.

Purpose of the Study:

• To address the underestimation of peripheral parameter uncertainty in sample size planning.
• To introduce new, robust, and simplified methods for power analysis.
• To restore conservatism and credibility to the sample size planning process.

Main Methods:

• Developed new methods for sample size planning within measured variable path analysis models.
• Demonstrated how deviations in peripheral parameter estimates can undermine statistical power.
• Applied methods to models including multiple linear regression and analysis of variance.

Main Results:

• Identified that imprecise estimates of contextual parameters significantly impact the power of focal parameters.
• Proposed a "hope-for-the-best-but-plan-for-the-worst" approach to power analysis.
• The new methods simplify sample size planning while enhancing robustness.

Conclusions:

• The proposed methods offer a more conservative and credible approach to sample size determination.
• Simplified power analysis procedures can improve the rigor of research designs.
• Addressing parameter uncertainty is vital for maintaining statistical power and research integrity.