Beyond "non-significant" results: Why and how to test for practical equivalence

Area of Science:

• Biostatistics
• Scientific Methodology
• Research Integrity

Background:

• Misinterpretation of P ≥ 0.05 as indicating "no effect" or "no difference" is prevalent in scientific literature.
• Statistically nonsignificant results do not confirm the absence of a difference but indicate insufficient evidence.
• This ambiguity arises from either a true lack of meaningful difference or an inability to detect it due to sample size or variability.

Purpose of the Study:

• To highlight equivalence testing as a robust framework for distinguishing between true null effects and inconclusive findings.
• To introduce the two one-sided tests (TOST) approach for formally assessing practical or clinical significance.
• To provide practical guidance and tools for researchers to implement equivalence testing.

Main Methods:

• Explanation of the logical framework of equivalence testing.
• Application of the two one-sided tests (TOST) for paired and unpaired comparisons.
• Introduction of a freely available online calculator for common equivalence tests.

Main Results:

• Equivalence testing provides a method to formally test if scientifically meaningful effects can be ruled out.
• The TOST approach distinguishes between practical equivalence and inconclusive statistical evidence.
• Demonstration of equivalence testing's utility in various comparative scenarios.

Conclusions:

• Equivalence testing offers a superior approach to interpreting nonsignificant findings compared to traditional null hypothesis significance testing.
• Adoption of equivalence testing can enhance the rigor and clarity of scientific reporting.
• Accessible tools like the TOST calculator can facilitate broader implementation of equivalence testing in research.