Crowds Replicate Performance of Scientific Experts Scoring Phylogenetic Matrices of Phenotypes

Area of Science:

• Biodiversity research
• Evolutionary biology
• Computational phylogenetics

Background:

• The Tree of Life project requires categorizing phenotypes for millions of species, a task exceeding expert capacity.
• Crowdsourcing offers a potential solution for large-scale data collection by nonexperts.
• The reliability of nonexpert crowds for complex tasks like phylogenetic character coding is often questioned.

Purpose of the Study:

• To investigate the feasibility and accuracy of using citizen scientists for large-scale phylogenetic character coding.
• To develop and test a method for assigning character complexity to optimize expert and nonexpert contributions.
• To assess the potential of crowdsourcing to enhance biodiversity knowledge for phylogenetic tree building.

Main Methods:

• A study involving over 600 nonexpert participants (citizen scientists) was conducted.
• Participants used images to identify anatomical similarities (homologies) across diverse species.
• A novel procedure was developed to predict character difficulty and assign tasks to experts and nonexperts.

Main Results:

• Nonexpert citizen scientists achieved an average accuracy of 82% in identifying anatomical homology from images.
• The developed procedure enabled crowds to produce matrices with over 90% of cells scored correctly.
• The method reduced the number of characters requiring expert scoring by 50%.

Conclusions:

• Citizen scientists can effectively contribute to phylogenetic character coding, significantly aiding the Tree of Life project.
• A hybrid approach, assigning easier tasks to crowds and complex ones to experts, optimizes data collection efficiency.
• While current preparation time is substantial, future automation could unlock the full potential of crowdsourcing for biodiversity research.