Geometric deep learning enables 3D kinematic profiling across species and environments

Area of Science:

• Animal behavior analysis
• Biomechanical research
• Computational neuroscience

Background:

• Accurate 3D whole-body movement analysis is crucial for understanding animal behavior.
• 2D tracking methods struggle with freely moving animals due to occlusions and appearance changes.

Purpose of the Study:

• To develop a robust 3D animal pose estimation system for diverse species and behaviors.
• To overcome limitations of existing 2D tracking methods in naturalistic environments.

Main Methods:

• DANNCE (Deep Attentive Neural Network for Comprehensive Estimation) utilizes projective geometry and convolutional neural networks.
• The system was trained and validated on a large dataset of nearly seven million frames of rodent videos and 3D poses.

Main Results:

• DANNCE accurately tracked dozens of anatomical landmarks on freely moving rats and mice in naturalistic settings.
• The system demonstrated robust performance across different species, including rat pups, marmosets, and chickadees.
• Quantitative profiling of behavioral lineage during development was achieved.

Conclusions:

• DANNCE provides a robust and versatile solution for 3D animal pose estimation.
• This technology enables more comprehensive and accurate studies of animal behavior across various species and developmental stages.