Temporal Upsampling of Depth Maps Using a Hybrid Camera

Area of Science:

• Computer Vision
• Robotics
• 3D Reconstruction

Background:

• Consumer depth cameras offer limited frame rates, hindering applications with fast motion.
• High-frame-rate video cameras are widely available but lack depth information.

Purpose of the Study:

• To develop a hybrid camera system for high-frame-rate depth map acquisition.
• To enable temporal interpolation of depth maps using auxiliary color images.

Main Methods:

• A novel algorithm was developed to reconstruct intermediate depth maps.
• Simultaneous estimation of scene flow was performed alongside depth reconstruction.
• The hybrid system integrates a high-frame-rate video camera and a low-frame-rate depth camera.

Main Results:

• The developed algorithm accurately reconstructs depth maps for fast, non-rigid motions.
• Scene flow estimation proved more precise than existing tracking-based and state-of-the-art methods.
• Experiments demonstrated the effectiveness on single and multiple object scenarios.

Conclusions:

• The hybrid camera and novel algorithm effectively address the limitations of current depth sensing technologies.
• This approach enables high-fidelity 3D motion capture for dynamic scenes.
• The method offers improved precision for scene flow estimation in complex scenarios.