Inconsistency in the analysis of morphological deformities in chironomidae (Insecta: Diptera) larvae

The incidence of morphological deformities of chironomid larvae as an indicator of sediment toxicity has been studied for decades. However, standards for deformity analysis are lacking. The authors evaluated whether 25 experts diagnosed larval deformities in a similar manner. Based on high-quality digital images, the experts rated 211 menta of Chironomus spp. larvae as normal or deformed. The larvae were from a site with polluted sediments or from a reference site. The authors revealed this to a random half of the experts, and the rest conducted the assessment blind. The authors quantified the interrater agreement by kappa coefficient, tested whether open and blind assessments differed in deformity incidence and in differentiation between the sites, and identified those deformity types rated most consistently or inconsistently. The total deformity incidence varied greatly, from 10.9% to 66.4% among experts. Kappa coefficient across rater pairs averaged 0.52, indicating insufficient agreement. The deformity types rated most consistently were those missing teeth or with extra teeth. The open and blind assessments did not differ, but differentiation between sites was clearest for raters who counted primarily absolute deformities such as missing and extra teeth and excluded apparent mechanical aberrations or deviations in tooth size or symmetry. The highly differing criteria in deformity assignment have likely led to inconsistent results in midge larval deformity studies and indicate an urgent need for standardization of the analysis.