Real-time digital dermatitis detection in dairy cows on Android and iOS apps using computer vision techniques

The aim of the study was to deploy computer vision models for real-time detection of digital dermatitis (DD) lesions in cows using Android or iOS mobile applications. Early detection of DD lesions in dairy cows is crucial for prompt treatment and animal welfare. Android and iOS apps could facilitate routine and early DD detection in cows' feet on dairy and beef farms. Upon detecting signs of DD, dairy farmers could implement preventive and treatment methods, including foot baths, topical treatment, hoof trimming, or quarantining cows affected by DD to prevent its spread. We applied transfer-learning to DD image data for 5 lesion classes, M0, M4H, M2, M2P, and M4P, on pretrained YOLOv5 model architecture using COCO-128 pretrained weights. The combination of localization loss, classification loss, and objectness loss was used for the optimization of prediction performance. The custom DD detection model was trained on 363 images of size 416 × 416 pixels and tested on 46 images. During model training, data were augmented to increase model robustness in different environments. The model was converted into TFLite format for Android devices and CoreML format for iOS devices. Techniques such as quantization were implemented to improve inference speed in real-world settings. The DD models achieved a mean average precision (mAP) of 0.95 on the test dataset. When tested in real-time, iOS devices resulted in Cohen's kappa value of 0.57 (95% CI: 0.49 to 0.65) averaged across the 5 lesion classes denoting the moderate agreement of the model detection with human investigators. The Android device resulted in a Cohen's kappa value of 0.38 (95% CI: 0.29 to 0.47) denoting fair agreement between model and investigator. Combining M2 and M2P classes and M4H and M4P classes resulted in a Cohen's kappa value of 0.65 (95% CI: 0.54 to 0.76) and 0.46 (95% CI: 0.35 to 0.57), for Android and iOS devices, respectively. For the 2-class model (lesion vs. non-lesion), a Cohen's kappa value of 0.74 (95% CI: 0.63 to 0.85) and 0.65 (95% CI: 0.52 to 0.78) was achieved for iOS and Android devices, respectively. iOS achieved a good inference time of 20 ms, compared to 57 ms on Android. Additionally, we deployed models on Ultralytics iOS and Android apps giving kappa scores of 0.56 (95% CI: 0.48 to 0.64) and 0.46 (95% CI: 0.37 to 0.55), respectively. Our custom iOS app surpassed the Ultralytics apps in terms of kappa score and confidence score.