Tri-Perspective View Decomposition for Geometry Aware Depth Completion and Super-Resolution.

Area of Science:

• Computer Vision
• 3D Scene Understanding
• Geometric Modeling

Background:

• Depth completion and super-resolution are vital for 3D scene understanding.
• Existing methods struggle with fine-grained 3D geometry due to reliance on 2D data or raw 3D point clouds.

Purpose of the Study:

• To introduce a novel framework, Tri-Perspective View Decomposition (TPVD), for explicit 3D geometry modeling.
• To improve depth completion and super-resolution tasks.

Main Methods:

• TPVD decomposes 3D point clouds into three 2D views for explicit 3D geometry modeling.
• TPV Fusion updates 2D features via recurrent 2D-3D-2D aggregation.
• Refinement heads with adaptive neighbor search enhance geometric consistency.

Main Results:

• The proposed TPVD framework achieves state-of-the-art performance on depth completion and super-resolution.
• Experiments conducted on multiple benchmark datasets, including newly introduced TOFDC and TOFDSR datasets.
• Demonstrated superior geometric consistency and accuracy compared to existing methods.

Conclusions:

• TPVD offers a significant advancement in reconstructing precise 3D geometry from RGB-D data.
• The framework effectively addresses limitations of previous approaches in capturing fine-grained scene details.
• The developed datasets facilitate further research in depth estimation from time-of-flight sensors.