An Occlusion-Aware Framework for Real-Time 3D Pose Tracking.

Area of Science:

• Computer Vision
• Robotics
• Machine Learning

Background:

• Random forest methods are prominent for 3D temporal tracking using depth images, suitable for textureless objects.
• Existing methods struggle with real-world occlusions and increasing outliers in deep occlusion scenarios.
• Current approaches lack robust handling of dynamic occlusions and fast motion compensation.

Purpose of the Study:

• To propose an occlusion-aware framework for real-time and robust 3D pose tracking from RGB-D images.
• To enhance random forest-based tracking (RFtracker) by integrating local refinement and online rendering-based occlusion handling.
• To improve robustness against dynamic occlusions and cope with fast motion and system delays.

Main Methods:

• An occlusion-aware framework anchored in RFtracker, featuring integrated local refinement for optimal regression.
• Online rendering-based occlusion handling to address dynamic occlusion challenges.
• A lightweight convolutional neural network-based motion-compensated (CMC) module for fast motion and system delay compensation.

Main Results:

• The proposed framework demonstrates improved performance in heavily-occluded scenes compared to RFtracker.
• Real-time performance is preserved while enhancing robustness against occlusions.
• The integrated local refinement and CMC module effectively address prediction inconsistencies and motion-related issues.

Conclusions:

• The developed framework offers a significant improvement for 3D pose tracking in challenging, occluded environments.
• The combination of local refinement, online occlusion handling, and CMC provides robust and real-time tracking capabilities.
• This approach advances the state-of-the-art in pose tracking for applications utilizing RGB-D data.