DPODv2: Dense Correspondence-Based 6 DoF Pose Estimation

Area of Science:

• Computer Vision
• Robotics
• Machine Learning

Background:

• Accurate 6 DoF object pose estimation is crucial for robotic manipulation and augmented reality.
• Existing deep learning methods often rely solely on RGB images, limiting their performance in certain conditions.

Purpose of the Study:

• To introduce DPODv2, a novel three-stage method for 6 DoF object detection using dense correspondences.
• To develop a unified deep learning network capable of processing multiple imaging modalities (RGB and Depth).
• To propose a new pose refinement technique based on differentiable rendering.

Main Methods:

• DPODv2 integrates a 2D object detector with a dense correspondence estimation network.
• A multi-view pose refinement method using differentiable rendering compares predicted and rendered correspondences.
• The network is designed to be modality-agnostic, accepting RGB or Depth inputs.

Main Results:

• RGB data excels in dense correspondence estimation.
• Depth data improves pose accuracy when 3D-3D correspondences are available.
• The combination of RGB and Depth modalities achieves the highest overall performance.

Conclusions:

• DPODv2 demonstrates excellent results across various datasets and data modalities.
• The method is fast, scalable, and adaptable to different imaging inputs and training data types.
• The proposed differentiable rendering-based refinement enhances pose consistency across multiple views.