Feature-based behavior coding for efficient exploratory analysis using pose estimation.

Area of Science:

• Behavioral Science
• Machine Learning
• Computer Vision

Background:

• Traditional behavioral coding is time-consuming, especially during exploratory research.
• Developing coding schemes for complex behaviors can be challenging.
• Existing methods may not fully leverage advances in pose estimation technology.

Purpose of the Study:

• Introduce Feature-Based Behavior Coding (FBBC) as an efficient exploratory analysis method.
• Present Behavior Senpai, an open-source software for FBBC implementation.
• Demonstrate FBBC's utility in classifying complex postures.

Main Methods:

• Utilizes pose estimation to detect keypoints in video data.
• Applies dimensionality reduction to create interpretable feature time series.
• Integrates automated feature extraction with manual clustering for analysis.

Main Results:

• FBBC transforms video data into efficient feature time series for analysis.
• A case study successfully classified complex postures using combined features.
• Behavior Senpai software facilitates automated feature extraction and analysis.

Conclusions:

• FBBC enhances flexibility in developing behavioral coding schemes.
• The method significantly reduces the time required for repetitive observational analysis.
• FBBC bridges traditional behavioral research with modern machine learning techniques, advancing behavioral science.