Deep Learning-Based Human Joint Localization Using mmWave Radar and Sequential Frame Fusion.

Area of Science:

• Biomedical Engineering
• Signal Processing
• Machine Learning

Background:

• Millimeter wave (mmWave) Frequency-Modulated Continuous Wave (FMCW) radar offers advantages over RGB and depth cameras for human monitoring, including operation in low-light conditions and enhanced privacy.
• Conventional FMCW radar techniques struggle with accurately capturing spatiotemporal features for human joint point detection.

Purpose of the Study:

• To develop a novel framework for precise human joint position estimation using FMCW radar data.
• To enhance the representation of spatiotemporal features for improved accuracy in human pose estimation.

Main Methods:

• A novel framework incorporating sequential frame concatenation for preprocessing to maintain spatiotemporal continuity.
• A deep learning architecture combining Convolutional Neural Network (CNN) for spatial features, Transformer for long-term dependencies, and Bidirectional Long Short-Term Memory (Bi-LSTM) for temporal modeling.

Main Results:

• The proposed approach effectively utilizes spatiotemporal data, leading to improved accuracy and reliability in human joint position detection.
• Experimental validation demonstrated a Mean Absolute Error (MAE) of 1.77 cm and a Root Mean Squared Error (RMSE) of 2.92 cm.

Conclusions:

• The novel framework significantly enhances the accuracy and reliability of human joint position estimation using FMCW radar.
• The integrated deep learning architecture optimizes spatiotemporal data processing, accelerating model convergence and reaffirming the model's effectiveness.