Ignoring what we don't know in accelerometer-based behaviour classification: the open-set recognition problem.

Area of Science:

• Animal behavior analysis
• Machine learning applications
• Bio-logging technology

Background:

• Supervised machine learning is widely used for classifying animal behaviors using accelerometer data.
• Current models assign data to predefined categories but fail to recognize novel behaviors, leading to overprediction of known classes.
• This limitation, known as open-set recognition, is an underexplored challenge in accelerometer-based behavior classification.

Purpose of the Study:

• To describe the open-set recognition problem in animal behavior classification using accelerometers.
• To assess four potential solutions for addressing this limitation.
• To provide recommendations for improving the reliability of behavior classification models.

Main Methods:

• Evaluation of a multiclass model with an 'other' category.
• Assessment of threshold-based models.
• Analysis of one-class models.
• Implementation and testing of binary one-versus-all models.

Main Results:

• Traditional multiclass models exhibit high false-positive rates when encountering behaviors not included in the training data.
• Binary one-versus-all models demonstrate a more conservative and reliable performance in open-set scenarios.
• The study highlights significant uncertainty in real-world applications of current classification methods.

Conclusions:

• Open-set recognition is a critical, yet often overlooked, issue in accelerometer-based animal behavior analysis.
• Binary one-versus-all models are recommended as a more robust approach, especially when focusing on specific behaviors.
• Increased awareness of this limitation is crucial for accurate interpretation of animal behavior data.