High-Resolution Open-Vocabulary Object 6D Pose Estimation

Area of Science:

• Computer Vision
• Robotics
• Artificial Intelligence

Background:

• 6D pose estimation for unseen objects remains a significant challenge.
• Existing Vision-Language Models (VLMs) show promise but underperform compared to traditional model-based methods.
• Natural language offers a flexible way to describe and identify objects for pose estimation.

Purpose of the Study:

• To introduce Horyon, an open-vocabulary VLM-based architecture for relative pose estimation between two scenes of an unseen object.
• To leverage textual prompts for object identification and feature extraction in pose estimation.
• To establish a new state-of-the-art in VLM-based 6D pose estimation for unseen objects.

Main Methods:

• Developed Horyon, an open-vocabulary VLM architecture utilizing textual prompts for scene understanding.
• Implemented a method to identify unseen objects using text prompts and extract high-resolution, multi-scale features.
• Utilized extracted features for cross-scene matching and registration to determine relative pose.

Main Results:

• Horyon achieved state-of-the-art performance across four diverse datasets (REAL275, Toyota-Light, Linemod, YCB-Video).
• The proposed method demonstrated superior results in 6D pose estimation for unseen objects.
• Outperformed previous best approaches by 12.6 in Average Recall, showcasing significant advancement.

Conclusions:

• Horyon represents a significant advancement in VLM-based 6D pose estimation for unseen objects.
• The architecture effectively utilizes natural language for accurate object identification and pose determination.
• This work sets a new benchmark for open-vocabulary pose estimation tasks.