Hierarchical Multimodal Knowledge Matching for Training-Free Open-Vocabulary Object Detection

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Open-Vocabulary Object Detection (OVOD) seeks to detect objects beyond predefined categories by utilizing pre-trained vision-language models.
• Current OVOD methods often rely on supervised learning, which is less effective for novel categories with limited data.
• There is a need for improved methods to represent and detect novel object categories in OVOD.

Purpose of the Study:

• To introduce a novel Hierarchical Multimodal Knowledge Matching (HMKM) method for improved Open-Vocabulary Object Detection (OVOD).
• To enhance the representation of novel categories using both object and fine-grained attribute knowledge.
• To provide a training-free, plug-and-play module for existing OVOD models.

Main Methods:

• HMKM utilizes object prototype knowledge derived from limited images for category representation.
• Attribute prototype knowledge is incorporated to capture fine-grained category distinctions.
• During inference, object and attribute knowledge are adaptively combined to match region features with categories.

Main Results:

• The proposed HMKM method significantly improves performance in detecting novel categories.
• Experiments show effectiveness across various backbone architectures and datasets.
• HMKM demonstrates superior performance compared to existing approaches in data-limited novel category detection.

Conclusions:

• HMKM offers an effective approach for Open-Vocabulary Object Detection, particularly in scenarios with limited data for novel categories.
• The method's ability to integrate object and attribute knowledge enhances category representation and matching.
• HMKM provides a versatile and easily integrable solution for advancing OVOD capabilities.