Recalling Unknowns Without Losing Precision: An Effective Solution to Large Model-Guided Open World Object Detection.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Open World Object Detection (OWOD) aims to detect unknown objects and learn incrementally, but current methods struggle with limited generic object knowledge.
• Large Vision Models (LVMs), like Segment Anything Model (SAM), offer rich generic knowledge beneficial for advancing OWOD.
• Existing OWOD methods are limited by training sets with few known objects, hindering comprehensive object perception.

Purpose of the Study:

• To leverage the Segment Anything Model (SAM) for Open World Object Detection (OWOD).
• To establish the first SAM-Guided OWOD baseline and address its inherent challenges.
• To propose a novel method, SAM-Guided Robust Open-world Detector (SGROD), for improved unknown object recall without sacrificing known object precision.

Main Methods:

• Developed a SAM-Guided OWOD baseline by employing SAM's segmentation capabilities.
• Introduced Dynamic Label Assignment (DLA) to mitigate noisy labels from SAM's class-agnostic nature.
• Implemented Cross-Layer Learning (CLL) and SAM-based Negative Sampling (SNS) to prevent precision degradation on known objects.

Main Results:

• The proposed SGROD method significantly enhances the recall of unknown objects by approximately 20%.
• SGROD maintains highly competitive precision on known objects, addressing a key challenge in SAM-guided OWOD.
• Experiments on public datasets validate the effectiveness of SGROD in improving OWOD performance.

Conclusions:

• SAM can be effectively utilized to advance Open World Object Detection.
• SGROD presents a robust solution to challenges posed by SAM in OWOD, improving both recall and precision.
• The developed methods offer a promising direction for future research in open-world recognition and incremental learning.