Visual Camera Localization by Globally Consistent Descriptor Learning and Combined Bundle Adjustment.

Area of Science:

• Computer Vision
• Robotics
• Machine Learning

Background:

• Accurate camera pose estimation and scene scale recovery are crucial for autonomous systems.
• Existing methods often struggle with dynamic scenes or lack comprehensive feature integration.

Purpose of the Study:

• To introduce a novel localization pipeline that integrates both hand-crafted and learned features at point and object levels.
• To simultaneously recover scene scale from RGB input.
• To improve camera pose estimation accuracy and robustness in static and dynamic environments.

Main Methods:

• Developed a siamese Globally Consistent Feature Descriptor Network (GCFDNet) for learned descriptor generation and relative pose estimation.
• Incorporated a spatial-to-temporal feature fusion module within GCFDNet for enhanced pose regression and scale estimation.
• Designed a novel loss function balancing descriptor similarity and distance for improved descriptor learning.
• Proposed a combined Bundle Adjustment (BA) optimization framework integrating hand-crafted features, learned descriptors, and semantic objects.

Main Results:

• GCFDNet demonstrated enhanced relative pose regression and accurate scene scale estimation.
• The combined BA framework achieved state-of-the-art localization performance across diverse datasets (EuRoC, ScanNet, 7 Scenes, TUM RGB-D, Bonn).
• The proposed approach outperformed existing methods in both static and dynamic scenes.

Conclusions:

• The novel localization pipeline effectively leverages multi-level features for robust camera pose estimation and scale recovery.
• GCFDNet and the combined BA framework represent a significant advancement in visual localization.
• The approach shows strong potential for real-world applications in robotics and augmented reality.