Making inference with messy (citizen science) data: when are data accurate enough and how can they be improved?

Area of Science:

• Ecology
• Data Science
• Conservation Biology

Background:

• Ecological data frequently contain measurement or observation errors.
• Increased reliance on citizen scientists and automated algorithms amplifies concerns about data quality.
• Limited practical guidance exists for data users and managers on acceptable error rates and efficient data improvement strategies.

Purpose of the Study:

• To present a generalizable framework for evaluating data quality and identifying remediation practices.
• To demonstrate this framework using trail camera images classified by crowdsourcing.
• To determine acceptable misclassification rates and optimal remediation for occupancy modeling.

Main Methods:

• Expert validation to estimate baseline classification accuracy.
• Simulation to assess occupancy estimator sensitivity to misclassification rates.
• Regression techniques to identify misclassification predictors and prioritize remediation.

Main Results:

• Over 93% of images were accurately classified, yet insufficient for distribution estimation at the <5% bias threshold.
• A screening model achieved >97% accuracy in predicting misclassified images.
• Occupancy models accounting for false-positive error improved inference, even with 30% misclassification.

Conclusions:

• Combining sensitivity analysis with error estimation provides efficient data remediation solutions.
• Screening models or occupancy models addressing false-positive error are effective strategies.
• This framework benefits big data initiatives and conventional studies by improving data quality assessment and management.