3D Object Localisation from Multi-View Image Detections.

Area of Science:

• Computer Vision
• 3D Reconstruction
• Geometric Modeling

Background:

• Accurate 3D object reconstruction is crucial for robotics and augmented reality.
• Existing methods often require depth sensors or dense point clouds.
• Leveraging 2D object detections from multiple views offers a promising alternative.

Purpose of the Study:

• To develop a novel method for estimating 3D object position and occupancy from 2D object detections in multi-view images.
• To reformulate the 3D reconstruction problem as fitting a quadric (ellipsoid) in 3D space.
• To provide a robust and efficient solution applicable to generic scenes.

Main Methods:

• Fitting 2D ellipses to object detection bounding boxes across multiple views.
• Estimating a 3D quadric using a closed-form solution in dual-space (minimum three views) or non-linear optimization (two views).
• Implementing data preconditioning and non-linear refinement for robustness against inaccurate bounding boxes.

Main Results:

• A closed-form solution for 3D quadric estimation is demonstrated with three views.
• A two-view solution is achieved via non-linear optimization and object shape constraints.
• The method shows robustness to inaccurate bounding boxes through data preconditioning and refinement.
• Successful application on synthetic and real-world datasets in challenging scenarios.

Conclusions:

• The proposed method effectively recovers 3D object position and occupancy using only 2D detections.
• The quadric-based approach offers a robust and versatile solution for multi-view 3D reconstruction.
• Demonstrated applicability in realistic scenarios highlights the method's potential for various computer vision tasks.