A descriptive analysis of inter- and intraobserver agreement of body condition scoring methods in dairy cattle

Body condition scoring is a tool for assessing a dairy cow's energy reserves and nutritional status. However, traditional methods are time consuming, labor intensive, and subject to observer bias, leading to variable results. The use of digital images or autonomous, camera-based technology (AUTO) may reduce subjectivity and labor. This study evaluated the interobserver agreement between on-site BCS and digital image-derived scores, along with intra- and interobserver agreement among image-based BCS, autonomous BCS technology, and various image-based modes. This study was conducted on a commercial dairy with 2 sites, where 3 observers collected on-site BCS (BCS_O). Simultaneously, lateral- and rear-view photographs were combined and duplicated into 2 photo sets, which 4 observers used to assign BCS. The BCS from the photo sets were designated as BCS_IMAGE1 and BCS_IMAGE2, with the most frequent score from each photo set labeled as BCS_MODE1 and BCS_MODE2, respectively. Additionally, AUTO BCS data collected 1 d prior, on the day of, and 1 d following the on-site evaluation were aggregated into a mode (BCS_AUTO), and further stratified by site. Cohen's weighted kappa, using Fleiss-Cohen weights, assessed both intra- and interobserver agreement. The interobserver agreement for BCS_O was 0.73 (95% CI: 0.59, 0.87) between observers 1 and 2 (n = 52) and 0.64 (95% CI: 0.51, 0.77) between observers 1 and 3 (n = 63). The agreement between BCS_O and image-based scores for observer 1 (n = 427) was 0.62 (95% CI: 0.56-0.67) and 0.63 (95% CI: 0.57-0.68) for BCS_IMAGE1 and BCS_IMAGE2, respectively. For observer 2 (n = 105), agreement was 0.41 (95% CI: 0.28-0.54) and 0.41 (95% CI: 0.27-0.55), and for observer 3 (n = 62), it was 0.56 (95% CI: 0.37-0.75) and 0.41 (95% CI: 0.20-0.63), respectively. Intraobserver agreement between BCS_IMAGE1 and BCS_IMAGE2 showed substantial agreement for observer 1 (n = 493; 0.80, 95% CI: 0.76, 0.84), moderate agreement for observer 2 (n = 493; 0.58, 95% CI: 0.52, 0.65), almost perfect agreement for observer 3 (n = 491; 0.81, 95% CI: 0.78, 0.84), and substantial agreement for observer 4 (n = 493; 0.70, 95% CI: 0.65, 0.75). Agreement between the image-based modes and AUTO ranged from low to moderate. Agreement between BCS_MODE1 (n = 316) and BCS_MODE2 (n = 338) with BCS_AUTO was low, at 0.18 (95% CI: 0.10-0.25) and 0.14 (95% CI: 0.07-0.20), respectively. Stratified by site, BCS_MODE1 (n = 181) and BCS_MODE2 (n = 192) showed moderate agreement with BCS_AUTO from site 1 at 0.65 (95% CI: 0.58-0.73) and 0.61 (95% CI: 0.53-0.69), respectively. However, agreement with BCS_AUTO from site 2 was lower for BCS_MODE1 (n = 135) and BCS_MODE2 (n = 146) at 0.06 (95% CI: 0.03-0.10) and 0.07 (95% CI: 0.03-0.10), respectively. These findings suggest that image-based BCS offers an alternative to traditional live methods, but observer variability indicates training likely requires fine-tuning. Additionally, the methods presented here could serve as a valuable tool for evaluating the effectiveness of interventions aimed at improving BCS training.