Disease Severity Estimates-Effects of Rater Accuracy and Assessment Methods for Comparing Treatments

Assessment of disease severity is required for several purposes in plant pathology; most often, the estimates are made visually. It is established that visual estimates can be inaccurate and unreliable. The ramifications of biased or imprecise estimates by raters have not been fully explored using empirical data, partly because of the logistical difficulties involved in different raters assessing the same leaves for which actual disease has been measured in a replicated experiment with multiple treatments. In this study, nearest percent estimates (NPEs) of Septoria leaf blotch (SLB) on leaves of winter wheat from nontreated and fungicide-treated plots were assessed in both 2006 and 2007 by four raters and compared with assumed actual values measured using image analysis. Lin's concordance correlation (LCC, ρ_c) was used to assess agreement between the two approaches. NPEs were converted to Horsfall-Barratt (HB) midpoints and were compared with actual values. The estimates of SLB severity from fungicide-treated and nontreated plots were analyzed using generalized linear mixed modeling to ascertain effects of rater using both the NPE and HB values. Rater 1 showed good accuracy (ρ_c = 0.986 to 0.999), while raters 3 and 4 were less accurate (ρ_c = 0.205 to 0.936). Conversion to the HB scale had little effect on bias but reduced numerically both precision and accuracy for most raters on most assessment dates (precision, r = -0.001 to -0.132; and accuracy, ρ_c = -0.003 to -0.468). Interrater reliability was also reduced slightly by conversion of estimates to HB midpoint values. Estimates of mean SLB severity were significantly different between image analysis and raters 2, 3, and 4, and there were frequently significant differences among raters (F = 151 to 1,260, P = 0.001 to P < 0.0001). Only on 26 June 2007 did conversion to the HB scale change the means separation ranking of rater estimates. Nonetheless, image analysis and all raters were able to differentiate control and treated-plot treatments (F = 116 to 1,952, P = 0.002 to P < 0.0001, depending on date and rater). Conversion of NPEs to the HB scale tended to reduce F values slightly (2006: NPEs, F = 116 to 276, P = 0.002 to 0.0005; and, for the HB-converted values, F = 101 to 270, P = 0.002 to 0.0005; 2007: NPEs, F = 164 to 1,952, P = 0.001 to P < 0.0001; and, for HB-converted values, F = 126 to 1,633, P = 0.002 to P < 0.0001). The results reaffirm the need for accurate and reliable disease assessment to minimize over- or underestimates compared with actual disease, and the data we present support the view that, where multiple raters are deployed, they should be assigned in a manner to reduce any potential effect of rater differences on the analysis.