Animal Pose Estimation Based on Contrastive Learning with Dynamic Conditional Prompts.

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Traditional animal pose estimation methods struggle with limited training data, high annotation costs, and non-rigid deformations.
• Integrating prior knowledge into pose estimation models remains a significant challenge.

Purpose of the Study:

• To develop a novel approach for animal pose estimation that overcomes data scarcity and annotation costs.
• To leverage multimodal learning, specifically integrating language and image data, for enhanced pose estimation accuracy.

Main Methods:

• Proposed dynamic conditional prompts incorporating prior knowledge from language modalities.
• Utilized a multimodal (language-image) collaborative training and contrastive learning model.
• Constructed dynamic conditional prompts by fusing text prompt templates and image feature conditional tokens, enhanced by a contrastive language-image pre-training model.

Main Results:

• The proposed method significantly improves the precision of key-point localization in animal pose estimation.
• Experimental results demonstrate enhanced average accuracy on the AP-10K and Animal Pose datasets.
• The language-image contrastive learning approach effectively synchronizes and strengthens image and text feature interactions.

Conclusions:

• Dynamic conditional prompts offer an effective way to integrate linguistic prior knowledge into animal pose estimation.
• The multimodal approach provides a robust solution to challenges posed by limited data and complex deformations.
• This work advances the field of animal pose estimation through innovative integration of language and vision modalities.