The role of automated feedback in training and retaining biological recorders for citizen science

The rapid rise of citizen science, with lay people forming often extensive biodiversity sensor networks, is seen as a solution to the mismatch between data demand and supply while simultaneously engaging citizens with environmental topics. However, citizen science recording schemes require careful consideration of how to motivate, train, and retain volunteers. We evaluated a novel computing science framework that allowed for the automated generation of feedback to citizen scientists using natural language generation (NLG) technology. We worked with a photo-based citizen science program in which users also volunteer species identification aided by an online key. Feedback is provided after photo (and identification) submission and is aimed to improve volunteer species identification skills and to enhance volunteer experience and retention. To assess the utility of NLG feedback, we conducted two experiments with novices to assess short-term (single session) and longer-term (5 sessions in 2 months) learning, respectively. Participants identified a specimen in a series of photos. One group received only the correct answer after each identification, and the other group received the correct answer and NLG feedback explaining reasons for misidentification and highlighting key features that facilitate correct identification. We then developed an identification training tool with NLG feedback as part of the citizen science program BeeWatch and analyzed learning by users. Finally, we implemented NLG feedback in the live program and evaluated this by randomly allocating all BeeWatch users to treatment groups that received different types of feedback upon identification submission. After 6 months separate surveys were sent out to assess whether views on the citizen science program and its feedback differed among the groups. Identification accuracy and retention of novices were higher for those who received automated feedback than for those who received only confirmation of the correct identification without explanation. The value of NLG feedback in the live program, captured through questionnaires and evaluation of the online photo-based training tool, likewise showed that the automated generation of informative feedback fostered learning and volunteer engagement and thus paves the way for productive and long-lived citizen science projects.