EPro-PnP: Generalized End-to-End Probabilistic Perspective-n-Points for Monocular Object Pose Estimation.

Area of Science:

• Computer Vision
• Deep Learning
• Robotics

Background:

• Perspective-n-Point (PnP) is crucial for 3D object localization from 2D images.
• End-to-end deep learning approaches face challenges in learning 2D-3D correspondences for PnP, especially with ambiguous poses.

Purpose of the Study:

• To propose EPro-PnP, a probabilistic PnP layer for general end-to-end pose estimation.
• To enable differentiable learning of 2D-3D correspondences by outputting a pose distribution on the SE(3) manifold.

Main Methods:

• EPro-PnP treats 2D-3D coordinates and weights as learnable intermediate variables.
• Minimizes KL divergence between predicted and target pose distributions.
• Generalizes previous PnP methods and incorporates an attention-like mechanism.

Main Results:

• EPro-PnP enhances existing correspondence networks, improving performance on the LineMOD 6DoF pose estimation benchmark.
• A novel deformable correspondence network using EPro-PnP achieves state-of-the-art accuracy on the nuScenes 3D object detection benchmark.

Conclusions:

• EPro-PnP offers a robust framework for end-to-end pose estimation.
• It facilitates novel network designs and advances the capabilities of PnP-based methods in computer vision.