Challenges for reproducibility in species delimitation

Species richness is a foundational metric for comparing biodiversity among clades and regions in ecology, evolution, and conservation. As the biodiversity crisis accelerates, taxonomists face increasing pressure to delimit and name species rapidly, often relying on automated or semi-automated methods that prioritize speed over thoroughness. Yet the reproducibility of species delimitation (the degree to which independent experts reach consistent conclusions given the same evidence) remains largely unquantified, and its consequences for estimates of species richness have never been assessed at the scale of an entire fauna. This gap is consequential: if species delimitation is highly variable among practitioners, then published species counts may reflect the idiosyncrasies of individual taxonomists as much as the true biological structure of diversity within and among clades. Here we evaluate the precision, or reproducibility, of species delimitation in Neotropical freshwater fishes, the most species-rich continental vertebrate assemblage. We provided identical morphological and molecular datasets for species representing four genera to 40 taxonomic experts (10 per genus), who were asked to delimit species using (1) their preferred analytical approaches and (2) standardized analytical outputs. Total variance in species delimitation was partitioned into variance attributable to data analysis and interpretation of results. Total discordance was high (35.0%), and although standardization of analytical methods reduced discordance, substantial variance remained (23.5%) due to interpretative differences. Deviations from modal species estimates were not explained by any of seven expert attributes assessed, including taxonomic experience, publication record, geographic location, taxonomic concepts, or analytical methods. These results demonstrate that species delimitation can be subject to considerable subjectivity, even among experienced taxonomists working with identical data. Improving the precision of species delimitation will require coordinated advances across the full taxonomic workflow, including greater standardization of data acquisition and analysis, and clearer interpretative frameworks that explicitly define the evidentiary thresholds required to recognize species boundaries. Community-wide adoption of transparent reporting standards, analogous to those developed in genomics and clinical research, would help expose the sources of interpretative disagreement and facilitate more consistent application of species concepts across taxa and research groups. Developing benchmark datasets and shared reference taxonomies, against which new delimitation hypotheses can be evaluated, represents a tractable near-term goal for the systematic community. Ultimately, however, reproducible taxonomy cannot be achieved through procedural standardization alone. High-quality revisionary taxonomy must be grounded in experienced character evaluation and homology assessment, concept delimitation, and contingent analytical judgment, skills that are developed over years of immersive engagement with natural history collections, primary literature, and fieldwork. The decline of training opportunities in classical systematics therefore poses a direct threat not only to taxonomic productivity but to taxonomic consistency. Continued investment in a well-trained community of systematists, supported by institutions, collections, and funding agencies, remains the most reliable foundation for consistent and accurate species delimitation. Our results underscore that biodiversity metrics widely used in ecology, conservation planning, and macroevolutionary research are sensitive to practitioner variation in ways that have not previously been quantified, and that addressing this variation requires both methodological reform and sustained commitment to systematic expertise.