Multi-level feature fusion and joint refinement for simultaneous object pose estimation and camera localization.

Area of Science:

• Computer Vision
• Robotics
• 3D Reconstruction

Background:

• Object pose estimation and camera localization are vital for robotics and augmented reality.
• Current methods struggle with algorithm universality (category-level pose and scene-independent localization).
• Distinct feature extraction requirements hinder joint optimization.

Purpose of the Study:

• To develop a unified RGB-D based framework for simultaneous category-level object pose estimation and scene-independent camera localization.
• To enable feature sharing between object pose estimation and camera localization tasks.
• To enhance the universality and accuracy of both tasks.

Main Methods:

• A unified framework with separate SLO-ObjNet (pose estimation) and SLO-LocNet (localization) branches.
• Three-level feature fusion modules and a shared loss function for cross-task feature learning.
• A confidence calculation process for object pose accuracy and object-level Bundle Adjustment (BA) for precision improvement.

Main Results:

• The unified framework outperforms existing methods in both object pose estimation and camera localization accuracy on multiple datasets (REAL275, LineMOD, ScanNet, etc.).
• The trained networks (SLO-LocNet, SLO-ObjNet) demonstrated universality by performing well on unseen datasets (7 Scenes, TUM RGB-D).
• Evaluated the positive impact of feature fusion, confidence calculation, and BA optimization on overall performance.

Conclusions:

• The proposed unified framework effectively addresses simultaneous object pose estimation and camera localization.
• The approach achieves high accuracy and demonstrates significant universality for real-world applications.
• Feature sharing, confidence assessment, and BA optimization are key contributors to the framework's success.