Crowd sourcing remote comparative lameness assessments for dairy cattle

Area of Science:

• Veterinary Medicine
• Animal Science
• Agricultural Technology

Background:

• Routine lameness assessments on dairy farms are infrequent and often underestimate prevalence, delaying diagnosis and treatment.
• Relative assessments, comparing individuals, are more accurate than absolute ones, suggesting a comparative approach for lameness scoring.

Purpose of the Study:

• To develop and validate a remote, comparative lameness assessment method using non-experienced crowd workers.
• To evaluate the reliability and agreement of crowd worker assessments with experienced assessors.
• To determine the optimal number of crowd workers for accurate lameness assessment.

Main Methods:

• Recruited crowd workers online to compare side-by-side videos of cows walking, rating lameness differences (-3 to 3).
• Conducted 11 tasks with 50 crowd workers per task, alongside 5 experienced cattle lameness assessors.
• Analyzed worker responses using data filtering and clustering, calculating interobserver reliability (intraclass correlation coefficient - ICC).

Main Results:

• Moderate to high interobserver reliability (ICC = 0.46–0.77) was found among crowd workers.
• Crowd worker assessments showed excellent agreement with experienced assessors (ICC = 0.89–0.91).
• Agreement with experts increased with 2–10 crowd workers, plateauing thereafter (ICC > 0.80).

Conclusions:

• The remote comparative lameness assessment method is fast, cost-effective, and reliable for commercial herds.
• This approach facilitates large-scale data collection for training automated lameness detection algorithms.
• Utilizing crowd workers offers a viable solution for improving early lameness diagnosis and management in dairy cattle.