Unlocking plant health survey data: An approach to quantify the sensitivity and specificity of visual inspections

Invasive plant pests and pathogens cause substantial environmental and economic damage. Visual inspection remains a central tenet of plant health surveys, but its sensitivity (probability of correctly identifying the presence of a pest) and specificity (probability of correctly identifying the absence of a pest) are not routinely quantified. As knowing sensitivity and specificity of visual inspection is critical for effective contingency planning and outbreak management, we address this deficiency using empirical data and statistical analyses. Twenty-three citizen scientist surveyors assessed up to 175 labelled oak trees for three symptoms of acute oak decline. The same trees were also assessed by an expert who has monitored these individual trees annually for over a decade. The sensitivity and specificity of surveyors was calculated using the expert data as the 'gold-standard' (i.e., assuming perfect sensitivity and specificity). The utility of an approach using Bayesian modelling to estimate the sensitivity and specificity of visual inspection in the absence of a rarely available 'gold-standard' dataset was then examined with simulated plant health survey datasets. There was large variation in sensitivity and specificity between surveyors and between different symptoms, although the sensitivity of detecting a symptom was positively related to the frequency of the symptom on a tree. By leveraging surveyor observations of two symptoms from a minimum of 80 trees on two sites, with reliable prior knowledge of sites with a higher (~0.6) and lower (~0.3) true disease prevalence we show that sensitivity and specificity can be estimated without 'gold-standard' data using Bayesian modelling. We highlight that sensitivity and specificity will depend on the symptoms of a pest or disease, the individual surveyor, and the survey protocol. This has consequences for how surveys are designed to detect and monitor outbreaks, as well as the interpretation of survey data that is used to inform outbreak management.