mmPose-NLP: A Natural Language Processing Approach to Precise Skeletal Pose Estimation Using mmWave Radars.

Area of Science:

• Computer Vision
• Signal Processing
• Machine Learning

Background:

• Skeletal key-point estimation is vital for applications like autonomous driving and healthcare.
• Traditional optical sensors struggle with poor lighting and adverse weather.
• Millimeter-wave (mmWave) radar offers a robust alternative due to its environmental resilience.

Purpose of the Study:

• To introduce mmPose-NLP, a novel sequence-to-sequence (Seq2Seq) model for skeletal key-point estimation using mmWave radar data.
• To demonstrate the first method capable of estimating up to 25 skeletal key points using only mmWave radar.
• To leverage Natural Language Processing (NLP) inspired techniques for radar data processing.

Main Methods:

• mmWave radar point-cloud (PCL) data are voxelized, analogous to NLP tokenization.
• A sequence-to-sequence architecture processes N frames of voxelized radar data.
• Skeletal key points are predicted as voxel indices, then converted to 3-D coordinates.

Main Results:

• The mmPose-NLP system achieves high accuracy, with localization errors under 3 cm across depth, horizontal, and vertical axes.
• The method successfully estimates up to 25 skeletal key points using mmWave radar data alone.
• Performance variations based on the number of input frames (N=1 to 10) were analyzed.

Conclusions:

• mmPose-NLP presents a significant advancement in skeletal key-point estimation using mmWave radar.
• The NLP-inspired approach offers robustness and precision, overcoming limitations of optical sensors.
• The study provides open-source code, facilitating further research in this domain.