Smart Multimodal In-Bed Pose Estimation Framework Incorporating Generative Adversarial Neural Network.

Area of Science:

• Computer Science
• Biomedical Engineering
• Artificial Intelligence

Background:

• In-bed pose estimation is crucial for applications like sleep disorder monitoring and pressure ulcer risk assessment.
• Existing methods often struggle with accuracy in diverse environments and under varying cover conditions.

Purpose of the Study:

• To propose a novel multimodal in-bed pose estimation framework using deep learning.
• To enhance pose estimation accuracy by integrating feature fusion and generative models.

Main Methods:

• Utilized a deep learning framework trained on the Simultaneously-collected multimodal Lying Pose (SLP) dataset.
• Employed long wave infrared (LWIR) and depth image modalities.
• Developed a feature fusion network and a generative model for RGB image synthesis.

Main Results:

• Achieved a high performance of 97.8% at PCKh @0.5, outperforming other fusion-based models.
• Demonstrated improved accuracy across different cover thicknesses and in both home and hospital environments.
• The generative model significantly enhanced the overall pose estimation accuracy.

Conclusions:

• The proposed multimodal deep learning framework offers a robust solution for in-bed pose estimation.
• The method is generalizable to various real-world scenarios, including those with occlusions.
• This technology has the potential to significantly advance telehealth and patient monitoring systems.