Optimizing Intermediate Representations: A Framework for Low-Cost, High-Accuracy Behavior Quantification.

Area of Science:

• Neuroscience
• Genetics
• Ethology
• Computer Vision
• Machine Learning

Background:

• Quantitative measurement of animal behavior is crucial in neuroscience, genetics, and ethology.
• Automated analysis using computer vision relies heavily on pose estimation, which requires large, labor-intensive datasets for training.
• The assumption that more detailed anatomical tracking improves classification accuracy is widely held but not well-verified.

Purpose of the Study:

• To benchmark intermediate representations for supervised mouse behavior classification.
• To determine the optimal trade-off between annotation cost and model performance.
• To evaluate the impact of keypoint density, temporal features, and segmentation-derived descriptors.

Main Methods:

• Systematic evaluation of classifier performance against varying keypoint densities.
• Analysis of the impact of temporal feature engineering, including FFT-based signal processing.
• Benchmarking segmentation-derived shape descriptors against explicit pose estimation.

Main Results:

• Classifier performance is robust to keypoint density variations; increased density offers negligible gains.
• Temporal features, particularly FFT-based signal processing, consistently improve model performance.
• Whole-body segmentation achieves performance parity with explicit pose estimation for most behaviors.

Conclusions:

• The "more is better" intuition regarding pose tracking complexity is challenged.
• Efficient behavioral analysis pipelines should prioritize dataset volume and temporal dynamics over complex anatomical keypoints.
• Segmentation-derived descriptors offer a viable, low-cost alternative to traditional pose estimation.