3D Point-Voxel Correlation Fields for Scene Flow Estimation

Area of Science:

• Computer Vision
• 3D Reconstruction
• Robotics

Background:

• Estimating 3D motion (scene flow) from point clouds is crucial for applications like autonomous driving and robotics.
• Existing methods struggle with large object displacements due to reliance on local correlations.
• There is a need for methods that capture both short- and long-range dependencies in point cloud data.

Purpose of the Study:

• To propose a novel method for accurate scene flow estimation from consecutive point clouds.
• To address the limitations of local correlation methods in handling large displacements.
• To develop a robust framework capable of capturing both fine-grained local and extensive long-range 3D motion patterns.

Main Methods:

• Introduced Point-Voxel Correlation Fields with distinct point and voxel branches to analyze local and long-range correlations.
• Utilized K-Nearest Neighbors search for precise local correlation extraction.
• Employed multi-scale voxelization and pyramid correlation voxels for modeling long-range correspondences.
• Developed the Point-Voxel Recurrent All-Pairs Field Transforms (PV-RAFT) architecture with an iterative estimation scheme.
• Proposed Deformable PV-RAFT (DPV-RAFT) incorporating Spatial and Temporal Deformations for enhanced adaptability and precision.

Main Results:

• The proposed Point-Voxel Correlation Fields effectively capture both local and long-range dependencies in point clouds.
• PV-RAFT and DPV-RAFT demonstrated superior performance in scene flow estimation compared to state-of-the-art methods.
• Experiments on FlyingThings3D and KITTI Scene Flow 2015 datasets validated the effectiveness of the proposed approach.

Conclusions:

• The novel Point-Voxel Correlation Fields offer a significant advancement in scene flow estimation from point clouds.
• The integration of local and long-range correlation analysis provides robustness against large object movements.
• The proposed methods, particularly DPV-RAFT, achieve state-of-the-art results, paving the way for more accurate 3D motion understanding in complex environments.