Video-based cattle behaviour detection for digital twin development in precision dairy systems

Area of Science:

• Computer Vision
• Animal Science
• Digital Agriculture

Background:

• Digital twins in dairy farming necessitate accurate behavioral data for effective modeling.
• Current methods for monitoring cow behavior can be labor-intensive and lack real-time precision.

Purpose of the Study:

• To develop and validate a video-based framework for detecting, tracking, and classifying individual cow behaviors in commercial barn settings.
• To enable the integration of precise behavioral data into dairy digital twin architectures.

Main Methods:

• Utilized YOLOv11 for cow detection and ByteTrack for identity tracking.
• Employed the TimeSformer model for behavior recognition, trained on augmented video data.
• Evaluated model performance using accuracy, macro-F1 score, and real-time throughput.

Main Results:

• Achieved 85.0% overall accuracy and a macro-F1 score of 0.84 for behavior classification.
• Real-time processing throughput of 22.6 fps was demonstrated on NVIDIA L4 hardware.
• Attention visualizations confirmed focus on biologically relevant anatomical regions.

Conclusions:

• The developed framework provides a robust behavioral perception and state-estimation component for dairy digital twins.
• Continuous, per-animal activity streams facilitate individualized nutrition, predictive health, and automated farm management.
• This system offers a practical foundation for scalable dairy digital twin applications.