Towards Single Camera Human 3D-Kinematics

Area of Science:

• Biomedical Engineering
• Computer Vision
• Biomechanics

Background:

• Markerless 3D kinematic estimation is crucial for diagnosing movement disorders.
• Current multi-step methods (pose detection then model fitting) are limited by errors and hardware requirements.
• There is a need for a streamlined, accurate, and accessible 3D kinematic estimation technique.

Purpose of the Study:

• To develop a novel, end-to-end deep learning approach for direct 3D human kinematic estimation (D3KE) from videos.
• To overcome limitations of existing multi-step markerless motion capture pipelines.
• To enable fast, accurate, and clinically applicable kinematic analysis.

Main Methods:

• Proposed a novel deep neural network for direct 3D human kinematic estimation (D3KE).
• Employed an end-to-end training strategy for the deep neural network.
• Evaluated D3KE performance against existing 2D and 3D markerless motion capture pipelines.

Main Results:

• D3KE significantly outperformed existing pipelines, reducing joint angle error by 35% (from 5.44 to 3.54 degrees).
• The proposed end-to-end deep learning approach demonstrated robustness.
• D3KE achieved video framerate speeds, indicating efficient processing.

Conclusions:

• D3KE provides a superior alternative to multi-step markerless kinematic estimation.
• The method is fast, accurate, and has the potential for clinical use.
• Future applications include clinical analysis using mobile devices.