MultiCam: On-the-fly Multi-Camera Pose Estimation Using Spatiotemporal Overlaps of Known Objects

Area of Science:

• Computer Vision
• Robotics
• Augmented Reality

Background:

• Multi-camera Augmented Reality (AR) systems require accurate camera pose estimation for integrated functionality.
• Traditional methods often rely on initial calibration or continuous marker tracking, which have limitations.

Purpose of the Study:

• To develop a novel, marker-less approach for constant dynamic camera pose estimation in multi-camera AR.
• To overcome limitations of marker-based tracking, such as markers needing to be within the camera's field-of-view (FoV).

Main Methods:

• Proposed a method for dynamic camera pose estimation using spatiotemporal Field-of-View (FoV) overlaps of known objects.
• Enhanced state-of-the-art object pose estimation to update a spatiotemporal scene graph, enabling relations between non-overlapping FoV cameras.
• Introduced a new multi-camera, multi-object pose estimation dataset with temporal FoV overlap.

Main Results:

• The proposed approach achieved superior camera pose accuracy compared to state-of-the-art methods on the YCB-V and T-LESS datasets in FoV overlapping scenarios.
• Demonstrated effectiveness on both existing and the newly introduced datasets.
• Validated the marker-less approach for AR applications.

Conclusions:

• The developed method provides effective marker-less, dynamic camera pose estimation for multi-camera AR applications.
• Leveraging spatiotemporal object overlaps offers a robust alternative to marker-based tracking.
• The approach enhances AR system capabilities by enabling seamless integration of information from multiple, potentially non-overlapping, camera views.