Partitioning variability in animal behavioral videos using semi-supervised variational autoencoders.

Area of Science:

• Neuroscience
• Computer Vision
• Behavioral Science

Background:

• Understanding brain function necessitates detailed behavioral analysis.
• Video data is increasingly used for behavioral measurements, requiring advanced computational tools.
• Existing methods for analyzing behavioral videos have limitations.

Purpose of the Study:

• To develop a new video analysis tool for extracting interpretable behavioral features.
• To combine supervised pose estimation with unsupervised dimensionality reduction.
• To improve the analysis of behavioral and neural data.

Main Methods:

• Developed a novel video analysis tool.
• Integrated supervised pose estimation (e.g., DeepLabCut) with unsupervised dimensionality reduction.
• Applied the tool to videos of head-fixed and freely moving mice.

Main Results:

• The tool produces interpretable, low-dimensional representations of behavior.
• Extracted behavioral features contain more information than pose estimates alone.
• Demonstrated utility across diverse mouse preparations and behavioral paradigms.

Conclusions:

• The developed tool enhances the extraction and interpretation of behavioral information from videos.
• The interpretable features facilitate downstream behavioral and neural analyses.
• This approach offers improved precision and interpretability compared to existing methods.