Learning end-to-end respiratory rate prediction of dairy cows from red, green, and blue videos

Respiratory rate (RR) is an important indicator of the health and welfare status of dairy cows. In recent years, progress has been made in monitoring the RR of dairy cows using video data and learning methods. However, existing approaches often involve multiple processing modules, such as region of interest (ROI) detection and tracking, which can introduce errors that propagate through successive steps. The objective of this study was to develop an end-to-end computer vision method to predict RR of dairy cows continuously and automatically. The method leverages the capabilities of a state-of-the-art Transformer model, VideoMAE, which divides video frames into patches as input tokens, enabling the automated selection and featurization of relevant regions, such as a cow's abdomen, for predicting RR. The original encoder of VideoMAE was retained, and a classification head was added on top of it. Further, the weights of the first 11 layers of the pre-trained model were kept, whereas the weights of the final layer and classifier were fine-tuned using video data collected in a tiestall barn from 6 dairy cows. Respiratory rates measured using a respiratory belt for individual cows were serving as the ground truth (GT). The evaluation of the developed model was conducted using multiple metrics, including mean absolute error (MAE) of 2.58 breaths per minute (bpm), root mean squared error (RMSE) of 3.52 bpm, root mean squared prediction error (RMSPE; as a proportion of observed mean) of 15.03%, and Pearson r of 0.86. Compared with a conventional method involving multiple processing modules, the end-to-end approach performed better in terms of MAE, RMSE, and RMSPE. These results suggest the potential to implement the developed computer vision method for an end-to-end solution, for monitoring RR of dairy cows automatically in a tiestall setting. Future research on integrating this method with other behavioral detection and animal identification algorithms for animal monitoring in a freestall dairy barn can be beneficial for a broader application.