Exploring Fine-Grained Visual-Text Feature Alignment With Prompt Tuning for Domain-Adaptive Object Detection.

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Domain-adaptive object detection (DAOD) aims to bridge the gap between labeled source domains and unlabeled target domains by reducing domain bias.
• Pretrained vision-language models (VLMs) show promise for improving detector generalizability, but existing methods face challenges in fine-grained cross-domain feature alignment due to paradigm discrepancies.
• Current solutions may overlook crucial aspects like relational reasoning in prompts and cross-modal interactions, hindering effective alignment.

Purpose of the Study:

• To explore fine-grained visual-text feature alignment in DAOD using prompt tuning.
• To introduce a novel framework, FGPro, designed to mitigate domain discrepancies and enhance cross-domain generalization.
• To improve the ability of object detectors to adapt to new, unlabeled domains.

Main Methods:

• FGPro employs a three-level framework focusing on prompt-level, model-level, and regularization strategies.
• At the prompt level, a learnable domain-adaptive prompt and a prompt relation encoder are utilized to capture intertoken semantic relations.
• At the model level, bidirectional cross-modal attention facilitates detailed interaction between visual and textual features, complemented by a prompt-guided cross-domain regularization strategy for disentangled information injection.

Main Results:

• FGPro demonstrates significant performance improvements in DAOD across four diverse cross-domain scenarios.
• Specific gains include +1.0% AP50 in Cross-weather, +1.2% AP50 in Simulation-to-real, +1.3% AP50 in Cross-camera, and +2.8% AP50 in Industry.
• These results validate the effectiveness of FGPro's fine-grained alignment approach in capturing domain-aware information.

Conclusions:

• The proposed FGPro framework effectively aligns fine-grained visual-text features across source and target domains.
• FGPro successfully enhances domain-adaptive object detection by addressing paradigm discrepancies and leveraging prompt tuning.
• The framework's ability to capture domain-aware information leads to substantial performance gains, outperforming existing methods.