Fusion Poser: 3D Human Pose Estimation Using Sparse IMUs and Head Trackers in Real Time.

Area of Science:

• Computer Science
• Robotics
• Biomedical Engineering

Background:

• Vision-based motion capture methods face occlusion and economic challenges, limiting their use in complex virtual reality (VR) environments.
• Inertial sensors offer an alternative but struggle with real-world spatial localization for VR applications.

Purpose of the Study:

• To develop a robust motion capture system, Fusion Poser, that overcomes the limitations of existing methods for VR.
• To accurately track both human pose and real-world location using a combination of inertial sensors and deep learning.

Main Methods:

• A novel bidirectional recurrent neural network with a convolutional long short-term memory layer was proposed for accurate human pose estimation.
• Fusion Poser integrates estimated joint poses with head-mounted display tracker data for real-world coordinate localization.
• The model was trained using public motion capture datasets and a newly created real-world dataset.

Main Results:

• The proposed deep learning model demonstrated higher accuracy and stability in pose estimation by preserving spatio-temporal properties.
• Fusion Poser achieved robust real-world location tracking by combining pose estimation with tracker data.
• The system showed superior performance, particularly for lower body poses like squats and bows.

Conclusions:

• Fusion Poser effectively addresses the limitations of inertial sensor-based motion capture for VR applications.
• The integration of deep learning with inertial and tracking sensors provides accurate and stable human pose and location tracking.
• This approach enhances immersion and usability in complex VR environments.