From IR Images to Point Clouds to Pose: Point Cloud-Based AR Glasses Pose Estimation

Area of Science:

• Computer Vision
• Robotics
• Machine Learning

Background:

• Accurate pose estimation is crucial for augmented reality (AR) applications.
• Existing methods often struggle with single-view or infrared imagery.
• 3D point clouds offer a robust intermediate representation for pose estimation.

Purpose of the Study:

• To propose novel algorithms for AR glasses pose estimation from single infrared images.
• To leverage 3D point clouds as an intermediate representation.
• To improve accuracy and reduce error compared to existing methods.

Main Methods:

• Developed two algorithms: 'PointsToRotation' (Deep Neural Network) and 'PointsToPose' (hybrid Deep Learning and voting).
• Utilized a semi-supervised trained point cloud estimator for single-image 3D point cloud generation.
• Generated a novel point cloud dataset using the HMDPose dataset.

Main Results:

• Achieved approximately 50% error reduction compared to CloudPose (another point cloud-based method).
• Reduced pose estimation error by approximately 96% compared to a state-of-the-art image-based method.
• Demonstrated effective AR glasses pose estimation from single infrared images.

Conclusions:

• The proposed methods offer significant improvements in AR glasses pose estimation accuracy.
• 3D point clouds are a viable and effective intermediate representation for this task.
• The developed algorithms advance the field of real-time AR system tracking.