Comparison of deep learning-based three-dimensional human pose estimation methods with motion capture for gesture...

Area of Science:

• Cognitive Science
• Computer Vision
• Human-Computer Interaction

Background:

• Human gestures are crucial for communication, extensively studied in psychology and cognitive science.
• Traditional 3D gesture analysis relies on motion capture (MoCap), which is expensive and can impede natural movement.
• Deep learning-based human pose estimation (HPE) offers a potential alternative for 3D gesture measurement using standard cameras.

Purpose of the Study:

• To evaluate the accuracy of different human pose estimation (HPE) methods for 3D gesture analysis.
• To compare monocular and stereo HPE techniques against optical motion capture (MoCap) for upper-body keypoint estimation.
• To assess the viability of HPE as a cost-effective and accessible tool for 3D gesture research.

Main Methods:

• Four HPE methods (two monocular, two stereo) were tested using video recordings of participants gesturing during speech.
• Upper-body keypoints (wrists, elbows, shoulders, face) were estimated and compared to data from an optical motion capture (MoCap) system.
• Euclidean distance was used as the error metric to quantify differences between HPE and MoCap data.

Main Results:

• Stereo HPE methods significantly outperformed monocular methods across all tested keypoints.
• The most accurate stereo HPE method achieved an average error of 49.4 mm compared to MoCap.
• A 75.4% overlap in 3D gesture space was observed between HPE and MoCap at a 50 mm voxel size, indicating high spatial agreement.

Conclusions:

• Stereo HPE methods provide sufficient accuracy for practical 3D gesture analysis, serving as a viable alternative to MoCap.
• These findings support the development of an accessible toolbox for 3D gesture measurement using HPE for researchers and non-experts.
• Advances in HPE pave the way for more cost-effective and user-friendly tools in gesture-related research.