Equivalence testing in pesticide risk assessment: evaluation and practical guidance for design, analysis and interpretation

Harmful pesticide effects exceeding specific protection goals (SPG) may go undetected in underpowered studies. Regulatory honeybee field studies have consistently failed to reach the statistical power required under European Food Safety Authority (EFSA) guidance, which may have contributed to the approval of high-risk substances. Therefore, EFSA advised a shift from testing the null hypothesis of 'no effect' (difference testing) to equivalence testing. Under this new approach, a pesticide is classified as 'low risk' if the null hypothesis that its effect exceeds the SPG can be rejected. For honeybees, the recommended SPG is a colony size reduction below 10%. Critics have argued that this framework requires excessive experimental effort to demonstrate pesticide safety and proposed an alternative equivalence test defining treatment effects relative to the lower bound of the 90%-control-group confidence interval. Using simulations mimicking a regulatory honeybee field study, we show that although the two equivalence tests share the same trade-off between false 'low-risk' and false 'high-risk' classifications, only EFSA's equivalence test reliably identifies pesticides with effects > SPG at α = 0.2. Our results show that increasing sample sizes beyond current practice is often necessary for reliable regulatory assessment. However, for pesticides with effect sizes ≤ 5%, sample size requirements remain lower than those implied by the power requirement of EFSA's former difference-test-based guidance. Moreover, covariate adjustment - through a model term or balanced colony allocation using anticlustering randomisation - can reduce sample size requirements without losing power and thus save costs. Finally, we provide practical guidance for anticlustering randomisation, equivalence testing and their implementation in R.