TG-OverLoCK for open-vocabulary detection with text-guided coarse-to-fine refinement

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Open-vocabulary object detection (OVD) aims to detect objects using text descriptions, balancing accuracy with computational efficiency.
• Existing OVD methods often increase computational cost with enhanced cross-modal reasoning for rare categories.
• A need exists for efficient OVD models that improve recognition of fine-grained and novel categories without prohibitive inference expenses.

Purpose of the Study:

• To introduce TG-OverLoCK, an efficient extension for open-vocabulary object detection.
• To improve the recognition of rare and fine-grained categories in OVD.
• To demonstrate the effectiveness of a selective, coarse-to-fine language-conditioned refinement strategy.

Main Methods:

• TG-OverLoCK employs an overview-to-focus pathway, concentrating refinement capacity selectively.
• A lightweight overview stage generates a Context-Text Prior (CTP) for top-down guidance.
• A deeper focus stage utilizes cross-attention and ContMix-MLP for text-conditioned feature adaptation within a frozen-CLIP OVD pipeline.

Main Results:

• TG-OverLoCK-B achieved 38.2 AP on LVIS and 29.0 Novel AP on COCO OVD zero-shot evaluations.
• The model operates at 35 FPS on a single RTX 4090, demonstrating practical inference speed.
• Compared to the OverLoCK-B baseline, TG-OverLoCK-B improved LVIS AP by 2.3 points with a modest increase in latency and VRAM usage.

Conclusions:

• TG-OverLoCK offers an effective architectural solution for improving OVD accuracy, particularly for challenging categories.
• The selective refinement strategy provides accuracy gains with manageable computational overhead.
• The model isolates the benefits of coarse-to-fine text guidance within a standard OVD framework.