Long-Range Augmented Reality with Dynamic Occlusion Rendering.

Area of Science:

• Computer Vision
• Augmented Reality

Background:

• Achieving realism in Augmented Reality (AR) applications necessitates accurate occlusion-based rendering.
• Dynamic occlusion reasoning for real objects in outdoor environments presents significant challenges, particularly regarding depth estimation for moving objects.

Purpose of the Study:

• To develop a fast and accurate method for dynamic occlusion reasoning in large-scale outdoor AR.
• To infer metric depth of real objects without relying on explicit scene modeling or specialized depth sensors like 3D-LiDAR.

Main Methods:

• Utilized instance segmentation on color image data to detect dynamic real-world objects.
• Employed a top-down terrain elevation model or a deep learning-based monocular depth estimation model to infer object distances.
• Implemented a low-latency real-time framework incorporating semantic object tracking and prediction for video frames.

Main Results:

• Successfully detected and inferred depth for dynamic objects in outdoor scenes.
• Achieved real-time performance for occlusion reasoning and rendering in a video-see-through AR framework.
• Demonstrated the method's direct extendibility to optical-see-through AR systems.

Conclusions:

• The proposed method enables fast and accurate dynamic occlusion reasoning for outdoor AR applications.
• It effectively infers object depth using readily available color imagery, reducing reliance on complex sensing hardware.
• The system enhances the realism and immersion of augmented reality experiences by improving occlusion rendering.