Identifying data anomalies in milk component measurements from partial-day milking records

Area of Science:

• Dairy Science
• Animal Genetics
• Data Analytics

Background:

• Accurate milk and milk component data are vital for genetic evaluations and herd management.
• Data recording errors can significantly impact the validity of these decisions.
• Previous methods assessed data consistency at the farm level, but detecting cow-day level anomalies remained a challenge.

Purpose of the Study:

• To introduce and evaluate a novel metric, individual-level intraclass correlations, for assessing data consistency at the cow-day level.
• To compare the performance of this new metric against existing anomaly-detection methods.
• To develop a robust approach for identifying suspect milk component records.

Main Methods:

• Development of individual-level intraclass correlations as a metric for cow-day data consistency.
• Evaluation of the new metric against three common anomaly-detection techniques.
• Implementation of a two-step approach for estimating percentile thresholds to flag outliers.
• Examination of the impact of data shuffling on anomaly detection.

Main Results:

• The individual-level intraclass correlation metric demonstrated superior performance in identifying anomalies compared to conventional univariate and multivariate methods.
• The proposed methods effectively detect suspect milk component records at the individual cow-day level.
• The study also analyzed the detrimental effects of data shuffling on anomaly detection accuracy.

Conclusions:

• Individual-level intraclass correlations offer a robust and practical tool for enhancing milk component data quality at the cow-day level.
• These methods are crucial for improving the accuracy of genetic evaluations and herd management decisions.
• The findings provide valuable insights for dairy data quality assurance protocols.