Feature-based behavior coding for efficient exploratory analysis using pose estimation

This paper introduces feature-based behavior coding (FBBC), an efficient method for exploratory analysis in behavioral research using pose estimation techniques. FBBC addresses the challenges of traditional behavioral coding methods, particularly in the exploratory stages of research when coding schemes are not yet well defined. By leveraging keypoint detection and dimensionality reduction, FBBC transforms video data into interpretable feature time series, enabling researchers to analyze diverse postural patterns more efficiently. Also presented is Behavior Senpai, an open-source software implementation of FBBC that integrates automated feature extraction with human insight. A case study demonstrates FBBC's ability to classify complex postures by combining multiple features and manual clustering. While the current iteration focuses on instantaneous posture classification, the framework shows potential for expansion to action classification. FBBC offers increased flexibility in developing coding schemes and reduces the time-consuming nature of repetitive observations. This approach represents a considerable advancement in behavioral research, bridging traditional methods with modern machine-learning techniques. As FBBC is adopted and refined, it will contribute to more comprehensive and insightful behavioral analyses across the psychological and behavioral sciences.