Comparison of a machine learning model with a conventional rule-based selective dry cow therapy algorithm for...

We trained machine learning models to identify IMI in late-lactation cows at dry-off to guide antibiotic treatment, and compared their performance to a rule-based algorithm that is currently used on dairy farms in the United States. We conducted an observational test characteristics study using a dataset of 3,645 cows approaching dry-off from 68 US dairy herds. The outcome variables of interest were cow-level IMI caused by all pathogens, major pathogens, and Streptococcus and Streptococcus-like organisms (SSLO), which were determined using aerobic culture of aseptic quarter-milk samples and identification of isolates using MALDI-TOF. Individual cow records were extracted from the farm software to create 53 feature variables at the cow and 39 at the herd level, which were derived from cow-level descriptive data, records of clinical mastitis events, results from routine testing of milk for volume, and concentrations of SCC, fat, and protein. Machine learning (ML) algorithms evaluated were logistic regression, decision tree, random forest, light gradient-boosting machine, naive Bayes, and neural networks. For comparison, cows were also classified according to a conventional rule-based algorithm that considered a cow as high risk for IMI if she had one or more high SCC (>200,000 cells/mL) tests or ≥2 cases of clinical mastitis during the lactation of enrollment. Area under the curve (AUC) and Youden's index were used to compare models, in addition to binary classification metrics, including sensitivity, specificity, and predictive values. The ML models had slightly higher AUC and Youden's index values than the rule-based algorithm for all IMI outcomes of interest. However, these improvements in prediction accuracy were substantially less than what we had considered necessary for the technology to be a worthwhile alternative to the rule-based algorithm. Therefore, evidence is lacking to support the wholesale use of ML-guided selective dry cow therapy at the moment. We recommend that producers wanting to implement algorithm-guided selective dry cow therapy use a rule-based method.