FERAL: A Video-Understanding System for Direct Video-to-Behavior Mapping

Area of Science:

• Ethology
• Computer Vision
• Machine Learning

Background:

• Quantitative animal behavior analysis requires segmenting continuous actions into discrete states.
• Manual annotation is time-consuming, subjective, and not scalable.
• Existing automated methods rely on pose estimation, which can be limited by tracking accuracy and discards visual data.

Purpose of the Study:

• To introduce FERAL (Feature Extraction for Recognition of Animal Locomotion), a supervised video-understanding toolkit.
• To enable direct mapping of raw video to frame-level behavioral labels, bypassing pose estimation.
• To provide a user-friendly, open-source solution for scalable behavioral quantification.

Main Methods:

• FERAL utilizes a supervised learning approach to process raw video data.
• It directly extracts features from video frames to predict behavioral labels.
• The toolkit bypasses the need for intermediate pose estimation steps.

Main Results:

• FERAL outperforms state-of-the-art pose- and video-based methods on benchmark datasets.
• It achieves superior performance with significantly less training data compared to existing tools.
• The system demonstrates generalization across different species, recording conditions, and behavioral complexities.

Conclusions:

• FERAL offers a robust and scalable solution for animal behavior quantification.
• It lowers the barrier to entry for complex behavioral analyses.
• The toolkit facilitates broader applications in both laboratory and field research.