mmYOLOH-p: A Clinically-Oriented mmWave-Based Human Pose Estimation Tool for Unobtrusive Patient Monitoring

Human pose estimation (HPE) identifies and locates keypoints on a person's body. Despite the effectiveness of various existing methods, there remains a gap in addressing specific requirements for real-world clinical applications of HPE. In this study, we propose mmYOLOH-p, a novel clinical-oriented HPE approach. Requirements for patient monitoring influenced our choice of sensing modality, data representation, and deep learning architecture. We used millimeter-wave (mmWave) radar because it is unobtrusive, privacy friendly and robust against poor lighting and weather conditions. A pre-processing pipeline was developed to generate 2D heatmap representations of radar data and ultimately overcome scalability issues related to sparse point cloud representations. Cross-modal adaptation of the YOLOv8-pose architecture was performed since it enables one-shot efficiency and can be extended to perform concurrent tasks, such as human activity recognition, while avoiding cascading latency. mmYOLOH-p outperformed HuPRNet, the current state of the art in heatmap-based HPE, in terms of total average precision (AP), per-joint AP, and inference time. The highest AP50, AP75, and AP50-95 scores achieved in the experimental evaluation were 99.1, 92.6, and 83.0 respectively. The results position mmYOLOH-p as a potentially viable solution for activity recognition and gait analysis.