SKDF: A Simple Knowledge Distillation Framework for Distilling Open-Vocabulary Knowledge to Open-World Object...

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Open World Object Detection (OWOD) extends classic object detection by requiring models to identify known and unknown objects.
• Vision-Language Models (VLMs) possess extensive open-world knowledge but are limited by text prompts for localization.
• Real-world scenarios often lack pre-defined language descriptions, hindering VLM application in OWOD.

Purpose of the Study:

• To adapt large pre-trained VLMs for OWOD tasks by distilling their knowledge into a language-agnostic detector.
• To address the challenge of unknown object detection without relying on text prompts.
• To improve the performance and robustness of OWOD systems in diverse, real-world conditions.

Main Methods:

• Knowledge distillation from VLMs to a language-agnostic detector.
• A down-weight training strategy to mitigate negative impacts on known object learning.
• Cascade decoupled decoders to separate localization and recognition learning processes.
• Development of the "IntensiveSet" benchmark for evaluating unknown object detection.

Main Results:

• Simple knowledge distillation surprisingly improved unknown object detection, even with limited data.
• The proposed down-weight training and cascade decoupled decoders effectively preserved known object learning.
• Experiments demonstrated the effectiveness of the proposed methods on OWOD, MS-COCO, and the new IntensiveSet benchmark.

Conclusions:

• The proposed methods successfully distill open-world knowledge for OWOD while preserving known object recognition.
• The techniques enhance the ability of detectors to handle both seen and unseen objects in complex environments.
• The IntensiveSet benchmark provides a valuable resource for advancing OWOD research, particularly for unknown object detection.