mmPrivPose3D: A dataset for pose estimation and gesture command recognition in human-robot collaboration using...

Area of Science:

• Robotics
• Computer Vision
• Human-Robot Interaction

Background:

• 3D pose estimation and gesture recognition are vital for safe human-robot interaction.
• RGB-D cameras, while effective, pose privacy risks due to visual data capture.
• Radar sensors offer a privacy-preserving alternative, outputting point-cloud data instead of images.

Purpose of the Study:

• To introduce mmPrivPose3D, a new dataset for 3D human pose and gesture recognition using radar data.
• To facilitate the development of machine learning algorithms for radar-based human-robot interaction.
• To address privacy concerns associated with traditional RGB-D camera-based systems.

Main Methods:

• Collected 3D radar point-cloud data using a single IWR6843AOPEVM radar sensor at 10 Hz.
• Synchronized radar data with 19 3D skeleton keypoints extracted from RGB-D images (Intel RealSense camera, 30 fps, Nuitrack SDK).
• Labeled the dataset with human gestures and movements from 15 participants.

Main Results:

• Successfully created mmPrivPose3D, a comprehensive dataset of synchronized radar point-cloud and 3D skeleton keypoint data.
• The dataset captures diverse human movements and gestures.
• Provides a foundational resource for advancing radar-based pose and gesture recognition.

Conclusions:

• The mmPrivPose3D dataset is a significant resource for privacy-preserving human-robot interaction.
• Enables research into machine learning models for accurate pose estimation and gesture recognition using radar.
• Paves the way for more secure and effective human-robot collaboration.