Advancing animal behavior recognition with self-supervised pre-training on unlabeled data

Deep learning-based animal activity recognition (AAR) achieves promising performance but remains constrained by its reliance on large labeled datasets. While pre-training offers a viable path toward reducing annotation dependency, existing approaches like transfer learning primarily utilize single-species labeled data, neglecting the substantial publicly available unlabeled data across species. To address this gap, this study introduces a contrastive learning based self-supervised learning framework that effectively leverages cross-species unlabeled data to mitigate annotation scarcity in AAR. The approach involves two stages: (1) Self-supervised pre-training of a patch time series Transformer (PatchTST) based encoder using cross-species unlabeled data with a time-frequency consistency objective to learn transferable motion representations; and (2) Fine-tuning on species-specific labeled data, where the pre-trained encoder serves as a feature extraction backbone in a novel classification model that integrates intra-axis local dynamics via PatchTST-based encoder and cross-axis global patterns via depth-wise separable convolution. Experimental results demonstrate that our approach significantly enhances recognition performance under limited labeled data, achieving improvements of 4.79% in accuracy and 4.57% in F1-score over a baseline trained from scratch, while also improving discrimination of semantically similar behaviors and maintaining robustness with reduced samples. This work establishes a new direction for scalable and label-efficient animal behavior monitoring.