LidPose: Real-Time 3D Human Pose Estimation in Sparse Lidar Point Clouds with Non-Repetitive Circular Scanning...

Area of Science:

• Computer Vision
• Robotics
• Sensor Fusion

Background:

• Human pose estimation is crucial for surveillance and human-computer interaction.
• Existing methods struggle with sparse and uniquely patterned lidar data, particularly from Non-Repetitive Circular Scanning (NRCS) sensors.
• Balancing spatial and temporal resolution is key for analyzing NRCS lidar data.

Purpose of the Study:

• To develop a novel, end-to-end pose estimation method for real-time human skeleton detection in NRCS lidar point clouds.
• To adapt existing vision-transformer architectures to handle the sparsity and scanning patterns of NRCS lidars.
• To create a comprehensive dataset for evaluating lidar-based human perception.

Main Methods:

• Proposed LidPose, a vision-transformer-based method building on ViTPose.
• Introduced adaptations to handle NRCS lidar data sparsity and scanning patterns.
• Implemented foreground/background segmentation for region of interest (RoI) selection.
• Utilized raw NRCS lidar measurement sequences for moving pedestrian detection and skeleton fitting.

Main Results:

• LidPose demonstrates effective real-time human skeleton estimation from NRCS lidar data.
• The method successfully addresses the challenges of data sparsity and scanning patterns.
• A novel, real-world, multi-modal dataset with 2D/3D skeleton ground truth was created for evaluation.

Conclusions:

• LidPose offers a robust solution for real-time human skeleton estimation using NRCS lidar.
• The developed method and dataset advance the field of lidar-based perception for surveillance.
• This work paves the way for improved pedestrian detection and analysis in challenging environments.