Pose estimation based on keypoints and monocular depth estimation for predicting cattle body weight and hip height.

Area of Science:

• Agricultural Engineering
• Computer Vision
• Animal Science

Background:

• Traditional computer vision systems (CVS) for predicting cattle body weight (BW) and hip height (HH) often rely on costly 3D imaging or impractical side-view 2D cameras.
• Top-down view 2D imaging combined with pose estimation offers a potential solution for extracting biometric features.
• Monocular depth estimation (MDE) can generate depth information from 2D images, further enhancing biometric analysis.

Purpose of the Study:

• To develop predictive models for BW and HH using features from body pose keypoints and MDE-generated depth images from top-down 2D infrared images.
• To compare the predictive performance of these models against models using features from 3D imaging systems.

Main Methods:

• Collected 395 top-down view videos of beef-on-dairy crossbred cattle using infrared and depth sensors.
• Utilized a pose estimation model to identify seven key anatomical landmarks.
• Applied zero-shot MDE to convert 2D infrared images into 3D representations and extracted features (volume, area, etc.).
• Processed depth images from a 3D imaging system using the same feature extraction pipeline.
• Evaluated Random Forest, Partial Least Squares Regression (PLS), and Support Vector Regression models using leave-one-block-out cross-validation.

Main Results:

• PLS models using keypoint-derived features achieved R2 of 0.90 for BW and 0.77 for HH.
• PLS models utilizing MDE-derived depth features demonstrated superior performance, achieving R2 of 0.95 for BW (RMSE 24.2 kg) and comparable results for HH.
• The predictive performance using 2D-derived features was comparable to that of 3D imaging systems.

Conclusions:

• Biometric features extracted from top-down 2D images, including those generated via MDE, provide effective and comparable predictions for cattle BW and HH.
• This approach presents a cost-effective and practical alternative to traditional 3D imaging systems for livestock monitoring and management.