Large-scale capture of hidden fluorescent labels for training generalizable markerless motion capture models

Area of Science:

• Ethology
• Biophysics
• Machine Learning

Background:

• Deep learning enables markerless animal behavior tracking, but models often lack generalizability.
• Manual data annotation is time-consuming, limits landmark labeling, and requires retraining for new environments.
• Current methods lead to idiosyncratic kinematic data analysis and sparse landmark tracking.

Purpose of the Study:

• To develop a novel approach, GlowTrack, for generating large-scale training data to improve the generalizability of animal behavior tracking models.
• To enable dense tracking of numerous landmarks in parallel, overcoming limitations of sparse manual annotation.
• To establish a foundation for standardized behavioral analysis pipelines.

Main Methods:

• A high-throughput method for generating hidden labels using fluorescent markers.
• A multi-camera, multi-light system to simulate varied visual conditions and enhance model robustness.
• A parallel landmark labeling technique for dense tracking applications.

Main Results:

• GlowTrack enables orders of magnitude increase in training data generation.
• The approach facilitates the development of models that generalize across different experimental setups and visual environments.
• Dense tracking of multiple landmarks becomes feasible, providing more comprehensive movement data.

Conclusions:

• GlowTrack significantly advances markerless animal behavior tracking by addressing model generalizability and data annotation limitations.
• The developed methods support the creation of standardized behavioral analysis pipelines.
• This work facilitates more thorough scrutiny of animal movement through dense, generalized tracking.