Practical guidelines for validation of supervised machine learning models in accelerometer-based animal behaviour...

Area of Science:

• Ecology
• Machine Learning
• Animal Behavior

Background:

• Supervised machine learning (ML) is increasingly used for animal behavior classification from accelerometer data.
• A lack of standardized protocols and validation standards hinders reliable application of ML in ecology.
• Overfitting, where models memorize training data instead of generalizing, is a significant challenge.

Purpose of the Study:

• To review validation practices in accelerometer-based animal behavior studies.
• To identify the prevalence of insufficient model validation.
• To propose guidelines for robust validation and overfitting detection in biologging.

Main Methods:

• Systematic review of 119 studies using supervised ML for animal behavior classification from accelerometers.
• Analysis of validation techniques employed in the reviewed literature.
• Theoretical overview of overfitting and its detection in animal accelerometry.

Main Results:

• 79% of reviewed studies (94 papers) exhibited insufficient model validation.
• Absence of independent test sets limits the interpretability and generalizability of findings.
• Potential for widespread overfitting in the field.

Conclusions:

• Standardized validation protocols are crucial for reliable ML in animal behavior analysis.
• Implementing rigorous validation techniques is essential to prevent and detect overfitting.
• Guidelines are provided to enhance the quality and reproducibility of ML-based ecological research.