Reconstructing 3D human pose and shape from a single image and sparse IMUs.

Area of Science:

• Computer Vision
• Human Motion Analysis
• Biomedical Engineering

Background:

• Model-based 3D pose estimation is crucial for human motion analysis.
• Vision-based and inertial-based methods have limitations like occlusion and drift.
• Existing hybrid methods face challenges with complex devices and slow convergence.

Purpose of the Study:

• To improve 3D human pose estimation accuracy.
• To develop a method fusing single images with sparse inertial measurement units (IMUs).
• To overcome limitations of existing vision-based, inertial-based, and hybrid approaches.

Main Methods:

• A dual-stream feature extraction network is employed.
• A model-attention network with a residual module fuses image and IMU data.
• 3D pose and shape parameters are obtained via a direct regression strategy.

Main Results:

• Reduced per vertex error (PVE) by 9.4 mm on the Total Capture dataset.
• Reduced mean per joint position error (MPJPE) by 7.8 mm on the Human3.6M dataset.
• Demonstrated effective fusion of sparse IMU data and images for improved pose accuracy.

Conclusions:

• The proposed method effectively fuses sparse IMU data and single images.
• Significant improvements in 3D human pose estimation accuracy were achieved.
• The approach offers a more robust and accurate solution for markerless human motion capture.