Evaluation of Commercial Camera-Based Solutions for Tracking Hand Kinematics

Area of Science:

• Biomechanics
• Human-Computer Interaction
• Robotics

Background:

• Markerless motion capture offers advantages in setup and usability over marker-based systems.
• Accurate hand kinematics tracking is crucial for clinical diagnostics and scientific research.
• The performance of popular markerless systems like Leap Motion Controller 2 (LM2) and MediaPipe (MP) during dynamic tasks requires further investigation.

Purpose of the Study:

• To evaluate and compare the accuracy of the Leap Motion Controller 2 (LM2) and MediaPipe (MP) markerless systems for capturing digit joint kinematics.
• To assess the influence of movement speed on the performance of these markerless systems.
• To benchmark the markerless systems against a gold-standard marker-based system (Vicon).

Main Methods:

• Eleven participants performed hand movement tasks at three speeds.
• Hand landmark data were simultaneously captured using LM2, MP, and a Vicon marker-based system.
• Digit joint angles were calculated, and root mean squared error (RMSE) and correlation coefficients (r) were computed relative to Vicon data.

Main Results:

• LM2 demonstrated significantly lower mean RMSE (14.8°) and higher mean correlation (r = 0.58) compared to MP (RMSE = 22.5°, r = 0.45).
• Increased movement speed significantly degraded MP performance (higher RMSE, lower r) but not LM2 performance.
• The proximal interphalangeal joint exhibited greater error but higher correlation across both systems.

Conclusions:

• LM2 provides more accurate real-time hand joint motion capture than MP, making it potentially more suitable for applications requiring high precision.
• While both systems capture joint-level motion, their accuracy may not meet the stringent requirements of all clinical and scientific applications.
• Movement speed is a critical factor affecting the accuracy of markerless hand tracking systems, particularly for MP.