The role of background variability for interpreting biological relevance and statistical significance in Collembola...

Soil mesofauna field studies conducted for ecological risk assessments of plant protection products (PPPs) require expertise for interpretation. The new statistical closure principle computational approach test (CPCAT; introduced for Poisson-distributed count data) is proposed as an alternative for established methods. However, the biological relevance of potential effects remains unclear. One aim was to investigate how to assess biological relevance. Biological control ranges (BCRs) were calculated for each taxon and sampling time point within individual studies and it was ascertained whether the average abundance of a treatment group is inside or outside the BCR. High control abundance variability of Collembola resulted in largely varying BCR values between studies. The second aim was to investigate performance of different statistical methods. Statistical analysis revealed that abundance data exhibited overdispersion in the majority of cases instead of Poisson distribution. Significant differences were observed in more than 50% of the comparisons using CPCAT, even before test item application happened in the field. Postapplication, most statistical significances determined by CPCAT occurred despite the absence of a rate-response relationship and the arithmetic mean abundance values in the assigned treatment plots being within the BCR; thus, they should be considered false-positive results. In contrast, the Dunnett's test and Ranked Dunnett's test, either considering normally distributed data or being independent from the data distribution, barely detected a significant difference at preapplication when the abundance average values of assigned treatment plots were within the BCR pre- and postapplication. In conclusion, prerequisites for CPCAT (Poisson distribution, no over- or underdispersion of data) do not apply in the majority of the cases examined here. Holistic interpretation of field data needs to consider the ecological relevance of observations (via comparison with the BCR), presence of a rate-response relationship, onset, scale, and pattern of the response, instead of solely focusing on statistics.