Advanced 3D Motion Prediction for Video Based Dynamic Point Cloud Compression.

Area of Science:

• Computer Vision
• Multimedia Compression
• Extended Reality

Background:

• Point cloud representation is crucial for immersive media and extended reality (XR) but suffers from high data rates.
• Current video-based point cloud compression (V-PCC) standards, like MPEG-I, face challenges due to the loss of 3D motion continuity in 2D projections, reducing coding efficiency.

Purpose of the Study:

• To develop advanced methods for estimating 2D motion vectors (MV) that preserve 3D object motion continuity within the V-PCC framework.
• To improve the inter-prediction coding efficiency of dynamic point clouds by enhancing motion vector prediction.

Main Methods:

• Proposed a general model to calculate 2D MVs using 3D motion and 3D-to-2D correspondence.
• Developed a geometry-based method for 2D MV estimation in the 2D attribute video using reconstructed 3D geometry.
• Introduced an auxiliary-information-based method for 2D MV estimation in both geometry and attribute videos using coarse 3D geometry.
• Implemented two approaches to utilize estimated 2D MVs: adding them to the motion vector candidate list and using them as additional centers for motion estimation.

Main Results:

• The proposed methods demonstrated significant coding gains compared to existing state-of-the-art motion prediction algorithms in V-PCC.
• Experimental results confirmed the effectiveness of both normative and non-normative integration of estimated 2D MVs.

Conclusions:

• The developed techniques effectively address the loss of 3D motion continuity in V-PCC, leading to improved compression performance.
• The proposed motion vector estimation and utilization strategies offer a promising solution for efficient dynamic point cloud compression in immersive media applications.