Differentiable Biomechanics Unlocks Opportunities for Markerless Motion Capture

Area of Science:

• Biomechanics
• Machine Learning
• Computer Vision

Background:

• Differentiable physics simulators accelerated on GPUs are emerging for machine learning.
• These simulators have potential for biomechanics research and markerless motion capture but are underutilized.
• Markerless motion capture requires accurate biomechanical models and inverse kinematics fitting.

Purpose of the Study:

• To demonstrate the application of differentiable physics simulators for markerless motion capture.
• To fit inverse kinematics and scale biomechanical models to individual anthropometric measurements using markerless data.
• To improve the accuracy of markerless motion capture through an end-to-end optimization approach.

Main Methods:

• Utilized differentiable physics simulators within a machine learning pipeline.
• Employed an end-to-end approach with implicit trajectory representation propagated through a forward kinematic model.
• Minimized reprojection error of 3D markers from images.
• Incorporated bundle adjustment for extrinsic camera parameter refinement and meta-optimization for model improvement.

Main Results:

• Achieved improved reprojection error compared to previous markerless motion capture methods.
• Produced accurate spatial step parameters, validated against an instrumented walkway.
• Demonstrated successful model scaling to individual anthropometric measurements.
• Showcased the utility of differential optimization for refining camera parameters and improving biomechanical models.

Conclusions:

• Differentiable physics simulators offer a powerful tool for markerless motion capture and biomechanics research.
• The proposed end-to-end method enhances accuracy and model fitting capabilities.
• This approach has potential applications in both control and clinical populations for motion analysis.