A three-dimensional virtual mouse generates synthetic training data for behavioral analysis

Area of Science:

• Computational neuroscience
• Animal behavior analysis
• Machine learning for biological data

Background:

• Accurate pose estimation is crucial for analyzing animal behavior.
• Manual annotation of behavioral data is time-consuming and prone to error.
• Synthetic data generation offers a potential solution to data scarcity and annotation challenges.

Purpose of the Study:

• To develop a novel method for generating realistic 3D synthetic animated mice.
• To utilize synthetic data for training and evaluating pose estimation models.
• To assess the utility of 3D model-based pose estimation for ethological classification.

Main Methods:

• Development of a 3D synthetic animated mouse model from computed tomography scans.
• Actuation of the model using animation and constrained joint movements to create synthetic behavioral data.
• Application of image-domain translation to generate realistic synthetic videos.
• Training of 2D and 3D pose estimation models using the synthetic dataset.
• Comparison of pose estimation accuracy and behavioral cluster definition between 2D and 3D methods.

Main Results:

• Synthetic animated mouse successfully generated realistic behavioral videos with ground-truth labels.
• Pose estimation models trained on synthetic data achieved accuracy comparable to models trained on manual datasets.
• 3D model-based pose estimation provided superior definition of behavioral clusters compared to 2D video analysis.
• The approach shows promise for facilitating automated ethological classification.

Conclusions:

• A 3D synthetic animated mouse model can effectively generate high-quality training data for pose estimation.
• Synthetic data-driven pose estimation improves the resolution of behavioral analysis.
• This methodology offers a scalable and reproducible approach for advancing automated behavioral phenotyping.