Related Experiment Video
Updated: Jun 30, 2026
05:41
A Step-by-Step Implementation of DeepBehavior, Deep Learning Toolbox for Automated Behavior Analysis
Published on: February 6, 2020
9.9K
Animal behavioral analysis and neural encoding with transformer-based self-supervised pretraining
Yanchen Wang1, Han Yu1, Ari Blau1
1Columbia University, New York, NY USA.
Arxiv
|March 11, 2026
Summary
We developed BEAST (Behavioral Analysis via Self-supervised pretraining of Transformers), a new framework for analyzing animal behavior from videos. It uses unlabeled data to improve neuroscience research, even with limited labels.
Area of Science:
- Neuroscience
- Computer Vision
- Machine Learning
Background:
- Understanding the brain requires studying behavior, but current video analysis methods need extensive labeled data.
- This limits behavioral analysis in neuroscience research, especially when labeled datasets are scarce.
Purpose of the Study:
- To introduce BEAST (Behavioral Analysis via Self-supervised pretraining of Transformers), a scalable framework for analyzing neuro-behavioral data.
- To leverage unlabeled video data for pretraining vision transformers for diverse behavioral analyses.
Main Methods:
- BEAST utilizes masked autoencoding and temporal contrastive learning on unlabeled video data.
- It pretrains experiment-specific vision transformers for behavioral analysis tasks.
Main Results:
- BEAST demonstrated improved performance across multiple species in three key tasks: behavioral feature extraction, pose estimation, and action segmentation.
- The framework showed effectiveness in both single- and multi-animal settings.
- BEAST accelerates behavioral analysis in data-scarce scenarios.
Conclusions:
- BEAST provides a powerful and versatile backbone model for behavioral analysis in neuroscience.
- The framework effectively addresses the challenge of limited labeled data in video-based behavioral studies.
- This approach enhances the ability to correlate neural activity with specific behaviors.
