Related Experiment Video
Updated: Nov 16, 2025
Estimation of Contact Regions Between Hands and Objects During Human Multi-Digit Grasping
Published on: April 21, 2023
A contactless method to measure real-time finger motion using depth-based pose estimation
Yean Zhu1, Wei Lu2, Weihua Gan3
1Bioengineering College of Chongqing University, Chongqing, China; School of Transportation and Logistics, East China Jiaotong University, Nanchang, China.
Background:
Finger mobility plays a crucial role in everyday living and is a leading indicator during hand rehabilitation and assistance tasks. Depth-based hand pose estimation is a potentially low-cost solution for the clinical and home-based measurement of symptoms of limited human finger motion.
Objective:
The purpose of this study was to achieve the contactless measurement of finger motion based on depth-based hand pose estimation using Azure Kinect depth cameras and transfer learning, and to evaluate the accuracy in comparison with a three-dimensional motion analysis (3DMA) system.
Methods:
Thirty participants performed a series of tasks during which their hand motions were measured concurrently using the Azure Kinect and 3DMA systems. We propose a simple and effective approach to achieving real-time hand pose estimations from single depth images using ensemble convolutional neural networks trained by a transfer learning strategy. Algorithms to calculate the finger joint motion angles are presented by tracking the locations of the 24 hand joints. To demonstrate their potential, the Azure-Kinect-based 3D finger motion measurement system and algorithms are experimentally verified through comparison with a camera-based 3DMA system, which is the gold standard.
Results:
Our results revealed that the Azure-Kinect-based hand pose estimation system produced highly correlated measurements of hand joint coordinates. Our method achieved excellent performance in terms of the fraction of good frames ( >80%) when the error thresholds were larger than approximately 2 cm, and the range of mean error distance was 0.23--1.05 cm. For joint angles, the Azure Kinect and 3DMA systems had comparable inter-trial reliability (ICC2,1 ranging from 0.89 to 0.97) and excellent concurrent validity, with Pearsons r-values >0.90 for most measurements (range: 0.88--0.97). The 95% BlandAltman limits of agreement were narrow enough for the Azure Kinect to be considered a valid tool for the measurement of all reported joint angles of the index finger and thumb in pinching. Moreover, our method runs in real time at over 45 fps.
Conclusion:
The results of this study suggest that the proposed method has the capacity to measure the performance of fine motor skills.
