The null need not be nil: Clarifying the parallel arbitrariness of difference testing and equivalence testing

Area of Science:

• Statistical analysis
• Hypothesis testing

Background:

• Determining the smallest effect size of interest is crucial in statistical analysis.
• Conflicting approaches in different statistical tests can cause confusion.

Purpose of the Study:

• To clarify the perceived arbitrariness of equivalence testing compared to difference testing.
• To highlight how nil null hypotheses in difference testing create an illusion of objectivity.

Main Methods:

• Comparative analysis of difference testing and equivalence testing methodologies.
• Examination of the role of the smallest effect size of interest in both approaches.

Main Results:

• Equivalence testing and difference testing share parallel arbitrariness.
• The explicit declaration of the smallest effect size of interest in equivalence testing contrasts with its implicit use in difference testing via nil null hypotheses.

Conclusions:

• The perception of equivalence testing being more arbitrary stems from the implicit nature of the smallest effect size in difference testing.
• Promoting good practice requires acknowledging and managing the inherent arbitrariness in both statistical testing methods.