Toward general object search in open reality.

Area of Science:

• Computer Vision
• Deep Learning
• Machine Learning

Background:

• Real-world scenarios are dynamic and open-ended, requiring deep models to adapt to general objects in open realities.
• Current computer vision models face challenges in General Object Search in Open Reality (GOSO) due to high scale variance and an ever-expanding set of unknown categories.

Purpose of the Study:

• To extend the General Object Search in Open Reality (GOSO) task.
• To address the challenges of scale variance and open-world recognition in GOSO.
• To propose a novel architecture for robust object search in open-world scenarios.

Main Methods:

• Formalization of the GOSO problem.
• Introduction of the Siamese Exchanged Attention Network (SEA-Net), featuring a novel branch with stage-stacked Siamese Exchanged Attention (SEA) layers and a Hierarchical Feature Fusion (HFF) module.
• Integration of an Open Score Fusion (OSF) module for robust matching scores during inference.
• Construction of two new evaluation benchmarks using COCO and LVIS datasets.

Main Results:

• SEA-Net demonstrates efficient scale adaptation and extracts matching-friendly deep features.
• The OSF module enhances matching robustness in open-world scenarios.
• Extensive experiments validate the effectiveness of the proposed SEA-Net architecture and benchmarks.

Conclusions:

• The proposed SEA-Net effectively addresses key challenges in General Object Search in Open Reality.
• The developed benchmarks provide suitable evaluation for the GOSO task.
• SEA-Net offers a promising solution for practical object recognition in dynamic, open-world environments.