Uses of uncertain statistical power: Designing future studies, not evaluating completed studies

Area of Science:

• Psychology
• Statistics
• Research Methodology

Background:

• Statistical power is central to methodological reforms aimed at enhancing the reliability of psychological research findings.
• The application of statistical power has varied, with some uses aligning with its definition and others diverging.
• New developments in power analysis address uncertainty stemming from sampling variability.

Purpose of the Study:

• To introduce a taxonomy of three distinct uses of statistical power: comparing procedure performance, designing studies, and evaluating completed studies.
• To review fundamental concepts of statistical power and recent quantitative advancements in power analysis.
• To provide practical guidance and tools for applying power analysis in research design.

Main Methods:

• Literature review of fundamental concepts and new quantitative developments in power analysis.
• Introduction of a taxonomy for classifying the uses of statistical power.
• Development of web applications and examples to illustrate power analysis for study design.

Main Results:

• Identified three primary uses of statistical power: comparing methods, designing studies, and evaluating completed studies.
• Demonstrated the utility of new quantitative developments in power analysis, particularly for study design.
• Highlighted the potential counterproductive nature of using power for evaluating completed studies.

Conclusions:

• Statistical power is best utilized in the design and planning phases of research.
• New methods and tools can improve the application of power analysis for robust study design.
• Recommendations are provided for the effective application of statistical power in substantive psychological research.