Sample-Size Planning for More Accurate Statistical Power: A Method Adjusting Sample Effect Sizes for Publication Bias...

Area of Science:

• Biostatistics
• Research Methodology
• Statistical Power Analysis

Background:

• Sample size determination for statistical power relies on population effect size, which is inherently unknown.
• Current practice often uses prior study's sample effect size as a proxy for population effect size.

Purpose of the Study:

• To highlight the inaccuracies of using raw sample effect sizes for power analysis.
• To introduce a novel method for adjusting effect size estimates to improve sample size planning.
• To provide accessible tools for researchers to implement the proposed methodology.

Main Methods:

• Demonstrated the tendency for underpowered studies when using unadjusted sample effect sizes.
• Developed and validated an alternative approach that corrects for bias and uncertainty in effect size estimates.
• Utilized several experimental designs to showcase the method's effectiveness.

Main Results:

• Unadjusted sample effect sizes frequently lead to significantly underpowered studies.
• The proposed bias and uncertainty adjustment method enhances the accuracy of sample size planning.
• The effectiveness of the adjusted method was confirmed across various experimental designs.

Conclusions:

• Relying on raw effect size estimates from previous studies for sample size calculation is unreliable and can lead to underpowered research.
• An adjusted approach accounting for bias and uncertainty provides more accurate sample size estimations.
• Open-source R package (BUCSS) and web applications are available to facilitate the adoption of these improved methods in research.