Unsupervised transfer learning enables multi-animal tracking without training annotation

Area of Science:

• Quantitative ethology
• Animal behavior analysis
• Deep learning applications

Background:

• Accurate tracking of animal locomotion is crucial for quantitative ethology.
• Current tracking methods often require laborious annotations and struggle with challenging conditions.

Purpose of the Study:

• To develop an unsupervised deep transfer learning method for multi-animal tracking (UDMT).
• To achieve state-of-the-art performance without requiring training annotations.

Main Methods:

• Utilizing a bidirectional closed-loop tracking strategy.
• Employing a spatiotemporal transformer network.
• Integrating modules for localization refining, identity correction, and automatic parameter tuning.

Main Results:

• UDMT accurately tracks multiple animals under challenging conditions like crowding, occlusion, and low contrast.
• Demonstrated versatility across five model organisms: mice, rats, Drosophila, C. elegans, and Betta splendens.
• Successfully combined with head-mounted microscopes for neuroethological studies.

Conclusions:

• UDMT offers a powerful, annotation-free solution for multi-animal tracking.
• The method enhances the ability to correlate animal locomotion with neural activity.
• UDMT advances quantitative ethology and neuroethological research.