Graph Neural Network and Spatiotemporal Transformer Attention for 3D Video Object Detection From Point Clouds

Area of Science:

• Computer Vision
• Robotics
• Machine Learning

Background:

• Current LiDAR-based 3D object detection primarily uses single-frame analysis.
• Temporal information in point cloud videos remains underexplored for enhancing detection accuracy.

Purpose of the Study:

• To develop a novel framework for 3D object detection in point cloud videos by exploiting temporal information.
• To improve the detection of small and moving objects by effectively encoding short-term and long-term motion patterns.

Main Methods:

• Proposed a Grid Message Passing Network (GMPNet) to encode short-term temporal patterns by treating grids of points as nodes in a k-NN graph.
• Introduced an Attentive Spatiotemporal Transformer GRU (AST-GRU) with Spatial and Temporal Transformer Attention modules to aggregate long-term frame information.
• Integrated the proposed modules into both online and offline 3D video object detection frameworks.

Main Results:

• Achieved state-of-the-art performance on the nuScenes benchmark for 3D object detection using point cloud videos.
• Demonstrated superior performance in detecting small and moving objects by effectively utilizing temporal motion cues.
• The method achieved top leaderboard ranking without requiring additional complex components.

Conclusions:

• Exploiting temporal information in point cloud videos is crucial for advancing LiDAR-based 3D object detection.
• The proposed GMPNet and AST-GRU effectively capture short-term and long-term motion patterns, leading to significant performance gains.
• The framework offers a robust and efficient solution for both online and offline 3D video object detection applications.