The importance of expert selection when identifying threatened ecosystems

Identifying threatened ecosystem types is fundamental to conservation and management decision-making. When identification relies on expert judgment, decisions are vulnerable to inconsistent outcomes and can lack transparency. We elicited judgements of the occurrence of a widespread, critically endangered Australian ecosystem from a diverse pool of 83 experts. We asked 4 questions. First, how many experts are required to reliably conclude that the ecosystem is present? Second, how many experts are required to build a reliable model for predicting ecosystem presence? Third, given expert selection can narrow the range opinions, if enough experts are selected, do selection strategies affect model predictions? Finally, does a diverse selection of experts provide better model predictions? We used power and sample size calculations with a finite population of 200 experts to calculate the number of experts required to reliably assess ecosystem presence in a theoretical scenario. We then used boosted regression trees to model expert elicitation of 122 plots based on real-world data. For a reliable consensus (90% probability of correctly identifying presence and absence) in a relatively certain scenario (85% probability of occurrence), at least 17 experts were required. More experts were required when occurrence was less certain, and fewer were needed if permissible error rates were relaxed. In comparison, only ∼20 experts were required for a reliable model that could predict for a range of scenarios. Expert selection strategies changed modeled outcomes, often overpredicting presence and underestimating uncertainty. However, smaller but diverse pools of experts produced outcomes similar to a model built from all contributing experts. Combining elicited judgements from a diverse pool of experts in a model-based decision support tool provided an efficient aggregation of a broad range of expertise. Such models can improve the transparency and consistency of conservation and management decision-making, especially when ecosystems are defined based on complex criteria.