A generalized approach for producing, quantifying, and validating citizen science data from wildlife images

Area of Science:

• Ecology
• Conservation Biology
• Citizen Science

Background:

• Professional researchers often doubt data quality from nonexpert citizen scientists.
• Citizen science offers potential to broaden ecological and conservation research scope and scale.

Purpose of the Study:

• To develop and validate a method for generating accurate, reliable ecological data using untrained citizen scientists.
• To assess the reliability of aggregated classifications from citizen science data for wildlife monitoring.

Main Methods:

• Over 28,000 volunteers classified 1.51 million images from a Serengeti camera-trap survey via www.snapshotserengeti.org.
• Classifications were aggregated using a plurality algorithm and validated against 3,829 expert-verified images.
• Three certainty metrics (evenness, fraction support, fraction blank) were developed to measure confidence in aggregated answers.

Main Results:

• Aggregated volunteer classifications agreed with expert data on 98% of images.
• Accuracy varied by species commonness, with rare species showing higher false positive/negative rates.
• Certainty metrics significantly predicted classification accuracy and can identify images for expert review.
• 90% of images were accurately classified with as few as 5 volunteers per image.

Conclusions:

• Citizen science, particularly the Snapshot Serengeti project, can produce reliable data for ecological monitoring.
• The developed certainty metrics effectively gauge the reliability of citizen science classifications.
• This approach, using aggregated volunteer input, provides a robust foundation for large-scale African wildlife monitoring.