Simultaneously-Collected Multimodal Lying Pose Dataset: Enabling In-Bed Human Pose Monitoring

Area of Science:

• Computer Vision
• Medical Imaging
• Healthcare Technology

Background:

• Computer vision algorithms struggle with adverse conditions like darkness and data limitations.
• In-bed human pose monitoring is crucial for healthcare but lacks suitable datasets for challenging environments.
• Existing methods are brittle in scenarios involving complete darkness or full occlusion.

Purpose of the Study:

• To introduce a novel multimodal dataset for in-bed human pose estimation.
• To address the limitations of existing datasets in adverse vision conditions.
• To facilitate the development of robust human pose estimation models for healthcare.

Main Methods:

• Collected a Simultaneously-collected multimodal Lying Pose (SLP) dataset with 109 participants.
• Utilized multiple imaging modalities: RGB, long wave infrared (LWIR), depth, and pressure map.
• Developed a physical hyperparameter tuning strategy for ground truth pose label generation.

Main Results:

• The SLP dataset enables effective training of state-of-the-art 2D pose estimation models.
• Achieved promising performance up to 95% PCKh@0.5 on a single modality.
• Demonstrated improved pose estimation performance by incorporating multiple modalities via a collaborative scheme.

Conclusions:

• The SLP dataset significantly advances in-bed human pose estimation capabilities.
• Multimodal data fusion enhances robustness and accuracy in adverse vision conditions.
• The dataset and methods pave the way for improved healthcare monitoring applications.