Citizen science: A new perspective to advance spatial pattern evaluation in hydrology.

Area of Science:

• Environmental science and hydrology
• Computational science and data analysis
• Citizen science and crowdsourcing

Background:

• Traditional scientific methods often rely on computer algorithms for spatial pattern analysis, which can be fast but may not always capture human perceptual nuances.
• Human intuition and visual reasoning are highly effective for tasks involving spatial pattern comparison, a capability often challenging for automated systems.
• Citizen science offers a scalable approach to harness collective human intelligence for scientific research, complementing computational methods.

Purpose of the Study:

• To investigate the effectiveness of citizen science in evaluating the reliability of algorithms designed for quantifying spatial pattern similarity.
• To compare human perception of similarity and dissimilarity in hydrological model outputs against advanced statistical performance metrics.
• To establish a benchmark dataset for testing novel spatial metrics using human classifications.

Main Methods:

• A citizen science project was established, engaging over 2500 volunteers to classify simulated spatial patterns from a hydrological catchment model.
• Volunteers provided more than 43,000 classifications for 1095 unique spatial pattern subjects.
• The study analyzed the capability of various statistical performance metrics to replicate human judgments of pattern similarity and dissimilarity.

Main Results:

• Human visual perception proved effective in rating similarity and dissimilarity between simulated spatial patterns.
• More complex statistical metrics did not necessarily outperform simpler ones in emulating human perception.
• Metrics varied significantly in their ability to unambiguously distinguish between similar and dissimilar patterns, highlighting the importance of this feature for metric reliability.

Conclusions:

• Citizen science provides a robust method for validating and benchmarking spatial analysis algorithms in hydrology.
• Human perception offers a crucial reference for assessing the performance of computational metrics for spatial pattern comparison.
• The developed dataset serves as a valuable resource for the scientific community to test and advance spatial metric development.