ROOT: Region-Word Alignment With Partial Optimal Transport for Open-Vocabulary Object Detection

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Open-vocabulary object detection (OVD) aims to identify novel objects using image-text pairs.
• Current OVD methods struggle with inaccurate region-word correspondences, leading to performance degradation.
• Existing strategies to reduce false matches often discard useful data.

Purpose of the Study:

• To introduce a novel framework, Region-word Alignment with Partial Optimal Transport (ROOT), for more accurate OVD.
• To address the limitations of current methods in handling noisy region-word alignments.
• To improve the robustness and effectiveness of mining region-word correspondences.

Main Methods:

• Reframing region-word matching as a partial distribution alignment problem using partial optimal transport.
• Generating an optimal transport plan for aligning region and word features.
• Calculating matching reliability scores to reweight contrastive alignment loss for enhanced accuracy.

Main Results:

• The ROOT framework significantly reduces misalignment errors in region-text matching.
• Valuable region-word correspondences are preserved, unlike in previous methods.
• Experiments on OV-COCO and OV-LVIS benchmarks show ROOT outperforms state-of-the-art approaches.

Conclusions:

• ROOT offers a more flexible and reliable approach to region-text matching in OVD.
• The partial optimal transport strategy effectively handles noise and improves alignment accuracy.
• The proposed method demonstrates significant advancements in open-vocabulary object detection capabilities.