'No Effect' Conclusions in Studies Reporting Nonsignificant Results Are Potentially Incorrect

Area of Science:

• Orthopaedic research
• Clinical study analysis
• Statistical inference

Background:

• Interpreting statistical significance in clinical studies is crucial.
• Claims of
• no effect
• require careful examination of reported effect sizes and confidence intervals.
• High-impact journals often publish studies with diverse statistical reporting.

Purpose of the Study:

• To analyze the spectrum of effect sizes associated with
• no effect
• claims in high-impact orthopaedic journals.
• To evaluate the asymmetry of confidence intervals (CIs) around the zero-effect estimate.

Main Methods:

• Screened articles from eight top-impact orthopaedic journals published in 2019 for
• no-effect
• statements.
• Calculated effect sizes (Cohen's d, ϕ, odds ratios, hazard ratios) for reported
• no effect
• findings.
• Assessed CI asymmetry using the ratio of tail lengths relative to zero.

Main Results:

• Analyzed 255 articles; median Cohen's d was 0.15, median ϕ was 0.06.
• Median estimate for odds/hazard ratios was 1.16.
• Median asymmetry ratio was 1.9, with 22% of studies showing ratios exceeding 5.

Conclusions:

• "No effect" statements encompass varied observed effects.
• Observed effects often lean towards a specific direction despite
• no effect
• conclusions.
• CI asymmetry frequently indicates a tendency in effect direction.