Related Experiment Video
Updated: Jun 9, 2026
Video Movement Analysis Using Smartphones (ViMAS): A Pilot Study
Published on: March 14, 2017
Interobserver agreement when using a four-point mobility scoring system to score dairy cattle from video footage
James P Wilson1, Vivi Thorup2, Nick J Bell3
1Department of Livestock and One Health, Institute of Infection, Veterinary and Ecological Sciences, University of Liverpool, Neston, UK.
Background:
Our observational study aimed to investigate the interobserver agreement between mobility observers of different research institutes.
Methods:
Interobserver agreement when using a four-point mobility score was assessed among 10 experienced observers. Video footage comprised of individual clips of 207 unique cows recorded while walking on nine different farms. Gwet's agreement coefficient (AC1) and Fleiss' kappa were calculated and reported to measure the agreement between observers.
Results:
Lameness (mobility score ≥ 2) prevalence estimates for the video footage sample varied from 36.2% to 57.0% between observers. The pairwise interobserver agreement (Gwet's AC1) ranged from 35.6% to 74.5%. The average observer agreement was 47.8%-66.9%, with three observers below 60%. Measuring across all observers, overall agreement was poor (Fleiss' κ = 0.28) despite their high level of experience.
Limitations:
Our study used a small pool of observers with considerable experience in mobility scoring. We also used videos to ensure consistency in the cows presented to the observers, which may not be reflective of real-world scoring conditions.
Conclusions:
Standardisation procedures for lameness outcome assessments (among researchers and by logical extension, assessors working in the field) urgently require further development.
