Disentangling data dependency using cross-validation strategies to evaluate prediction quality of cattle grazing...

Area of Science:

• Animal Science
• Agricultural Technology
• Machine Learning Applications

Background:

• Wearable sensors offer real-time monitoring of cattle feeding behavior in grazing systems.
• Machine learning (ML) techniques are increasingly used for analyzing sensor data.
• Data cross-validation (CV) is essential for evaluating predictive model performance but can be misleading if not properly applied.

Purpose of the Study:

• To evaluate the impact of different cross-validation (CV) strategies on predicting cattle grazing activity using wearable accelerometer data and ML algorithms.
• To assess the performance of Elastic Net Generalized Linear Model (GLM), Random Forest (RF), and Artificial Neural Network (ANN) under various CV methods.

Main Methods:

• Collected 3-axis accelerometer data from six Nellore bulls over 15 days, classifying behavior as grazing or not-grazing.
• Employed three CV strategies: holdout, leave-one-animal-out (LOAO), and leave-one-day-out (LODO) for GLM, RF, and ANN models.
• Trained algorithms on similar dataset sizes across all CV strategies.

Main Results:

• Machine learning techniques (RF and ANN) outperformed GLM in prediction accuracy.
• The holdout CV strategy yielded the highest nominal accuracy but likely due to lack of data independence.
• LOAO and LODO strategies provided more realistic performance estimates, with ANN showing a slight advantage over RF.

Conclusions:

• The choice of CV strategy significantly influences the perceived accuracy of predictive models for cattle behavior.
• Holdout validation can overestimate model performance due to data dependencies and carry-over effects.
• Selecting a CV strategy that mirrors real-world application scenarios (e.g., predicting for new animals or management) is critical for reliable model deployment.