Comparative evaluation of machine-learning algorithms for early detection of metritis in dairy cows

Metritis is a common postpartum uterine disease in dairy cows that impairs fertility, reduces milk yield, increases culling risk, and causes substantial economic loss. This study aimed to develop and externally validate machine learning models for early metritis prediction using routinely recorded calving and reproductive data from commercial Holstein dairy herds. The analytical unit was the calving-event record, so individual cows could contribute repeated observations across lactations and calendar years. A retrospective development dataset included 92,662 calving-event records collected from 2008 to 2021, with 12,754 metritis-positive and 79,908 metritis-negative records. An independent external validation dataset included 51,377 calving-event records collected from 2011 to 2017, with 5,650 metritis-positive and 45,727 metritis-negative records. Metritis was defined using standardized postpartum diagnostic criteria and recorded by farm veterinarians during routine herd-health monitoring. Eleven routinely available predictors were considered in the development dataset: parity, age at first calving, gestation length, dry period length, calving season, calving year, dystocia, retained placenta, twin status, calf sex, and calf birth weight. Data preprocessing included duplicate removal, complete-case deletion, categorical encoding, and Z-score normalization. The development dataset was split into training (80%) and independent test (20%) sets using stratified sampling. Class imbalance was addressed using the synthetic minority over-sampling technique applied only within the training data. Six algorithms were evaluated: logistic regression, decision tree, random forest (RF), extreme gradient boosting, light gradient boosting machine, and a stacking ensemble model with RF as the meta-learner. Hyperparameters were tuned using grid search with 5-fold cross-validation. Performance was evaluated using accuracy, precision, recall, F₁ score, area under the receiver operating characteristic curve (ROC-AUC), area under the precision-recall curve (PR-AUC), confusion matrices, calibration metrics, and bootstrap-based 95% CI. The stacking ensemble achieved the best overall internal performance, with an accuracy of 88.7%, ROC-AUC of 0.94, PR-AUC of 0.94, and the most favorable calibration profile (Brier score = 0.09; calibration slope = 0.99; calibration intercept = 0.0002). Error analysis showed a higher false-negative than false-positive rate, with missed cases more common among primiparous records, shorter dry periods, and records lacking strong overt risk indicators such as retained placenta or dystocia. Lowering the decision threshold from 0.50 to 0.48 modestly improved case detection. For external validation, all models were retrained on the 9 predictors available in both herds because dystocia and retained placenta were unavailable in the external dataset. Despite this reduced predictor set, the stacking ensemble remained the best-performing model, achieving an accuracy of 0.85, ROC-AUC of 0.89, and PR-AUC of 0.92, indicating strong generalizability. Retained placenta, calf birth weight, dystocia, twinning, and parity were the most influential predictors. These findings show that stacking-based machine learning models can provide accurate and robust early prediction of metritis using routine farm records and may support targeted monitoring and intervention in dairy herd management.