VPU: A Video-Based Point Cloud Upsampling Framework.

Area of Science:

• Computer Vision
• 3D Data Processing
• Geometric Deep Learning

Background:

• Point clouds are sparse, irregular 3D data crucial for many applications.
• Upsampling point clouds aims to increase their density and detail.
• Existing methods often process frames independently, missing temporal information.

Purpose of the Study:

• To develop a novel video-based point cloud upsampling framework (VPU).
• To effectively leverage temporal dependencies across consecutive point cloud frames.
• To improve the inference of local geometry distributions in upsampled point clouds.

Main Methods:

• Proposes a patch-based framework (VPU) for video-based point cloud upsampling.
• Introduces a spatio-temporal aggregation (STA) module to extract and align features from consecutive frames.
• STA module summarizes spatio-temporally consistent knowledge without complex motion estimation.

Main Results:

• The VPU framework significantly enhances point cloud upsampling performance.
• The STA module effectively infers local geometry by aggregating multi-frame information.
• Demonstrates substantial performance improvements over single-frame upsampling methods on benchmark datasets.

Conclusions:

• The proposed VPU framework effectively exploits temporal dependencies for superior point cloud upsampling.
• The STA module offers a robust and adaptable approach for integrating temporal information.
• This method provides a significant advancement in processing dynamic 3D point cloud data.