Statistical conclusion validity of rehabilitation research. A quantitative analysis

Area of Science:

• Rehabilitation Medicine
• Biostatistics
• Research Methodology

Background:

• The effectiveness of rehabilitation procedures is crucial for patient outcomes.
• Ensuring the statistical validity of research findings in rehabilitation is paramount.
• Previous reviews have not comprehensively assessed the statistical conclusion validity of rehabilitation studies.

Purpose of the Study:

• To evaluate the statistical conclusion validity of 100 data-based studies on rehabilitation effectiveness.
• To determine the statistical power of hypothesis tests within these rehabilitation studies.
• To identify potential biases, such as Type II errors, in the existing rehabilitation research literature.

Main Methods:

• Systematic review and analysis of 100 empirical studies on rehabilitation effectiveness.
• Computation of post hoc power coefficients for statistical hypothesis tests.
• Assessment of the median power to detect small, medium, and large effect sizes.

Main Results:

• The median statistical power to detect small effect sizes was critically low at 0.08.
• Median power for medium and large effect sizes was 0.26 and 0.56, respectively.
• These power levels suggest a substantial risk of Type II errors in the reviewed rehabilitation studies.

Conclusions:

• Rehabilitation research may be prone to a high rate of Type II errors, leading to potentially false-negative conclusions.
• The findings underscore the need for enhanced statistical power in future rehabilitation research designs.
• Improving research methodology can lead to more accurate and reliable evidence-based rehabilitation practices.