A Confidence-Based Multibody Kinematics Optimization for Markerless Motion Capture: A Proof of Concept

Area of Science:

• Biomechanics
• Computer Vision
• Motion Capture

Background:

• Markerless motion capture typically triangulates 3D points from 2D keypoints and uses multibody kinematics optimization (MKO).
• Existing methods often overlook 2D confidence heatmaps from pose estimation networks, limiting robustness to missing data.

Purpose of the Study:

• To develop and evaluate a novel MKO approach that incorporates 2D confidence heatmaps.
• To enhance the robustness and accuracy of markerless motion capture, particularly in scenarios with occluded or missing keypoints.

Main Methods:

• Modeled 2D confidence heatmaps as 2D Gaussian functions.
• Maximized the sum of modeled confidences by projecting the biomechanical model into camera planes.
• Evaluated the method on sit-to-stand, walking, and manual material handling tasks using a two-camera setup.

Main Results:

• Gaussian modeling of heatmaps showed high validity with a mean absolute difference of 0.011 compared to discrete maps.
• Confidence-based MKO yielded 3D joint positions and angles comparable to classical distance-based methods.
• The confidence-based method successfully computed 100% of frames, overcoming occultations that prevented computation in 89.3% of frames for the distance-based method.

Conclusions:

• Incorporating 2D confidence heatmaps into MKO significantly improves markerless motion capture robustness.
• The proposed method is particularly effective under challenging conditions, such as sparse camera setups and missing keypoints.
• This approach offers a promising advancement for markerless motion capture, maximizing information from pose estimation networks.