Measuring the accuracy of management effectiveness evaluations of protected areas

Evaluating the effectiveness of protected area management to help improve management outcomes is becoming an increasingly common practice. The evaluation tools developed and implemented in over 100 countries generally rely on the expert judgements of protected area managers. Despite the growing collection and use of management effectiveness evaluation data, there have been no previous attempts to measure the accuracy of these data. We measured the accuracy of managers' judgements about the conditions in their reserves by collecting independent field data. We also assessed how accurately the evaluation tool reflected managers' views by conducting semi-structured interviews with 23 protected area managers from New South Wales, Australia. We found that managers made highly accurate judgements of the extent of a common weed species, Rubus fruticosus (blackberry), but often misinterpreted the scope, scale and timeframe of the evaluation. These framing effects can lead to error being introduced into the evaluation dataset, affecting the precision of evaluations such that they cannot be reliably compared among reserves. We suggest that the wording of evaluation questions needs to be explicit about the assessment frame to minimize the influence of framing effects on management effectiveness evaluations.