Related Experiment Video
Updated: Aug 16, 2025
Capturing Dynamic Finger Gesturing with High-resolution Surface Electromyography and Computer Vision
Published on: March 28, 2025
Automatic detection of abnormal hand gestures in patients with radial, ulnar, or median nerve injury using hand pose
Fanbin Gu1, Jingyuan Fan1, Chengfeng Cai2
1Department of Microsurgery, Orthopedic Trauma and Hand Surgery, The First Affiliated Hospital, Sun Yat-sen University, Guangzhou, China.
Background:
Radial, ulnar, or median nerve injuries are common peripheral nerve injuries. They usually present specific abnormal signs on the hands as evidence for hand surgeons to diagnose. However, without specialized knowledge, it is difficult for primary healthcare providers to recognize the clinical meaning and the potential nerve injuries through the abnormalities, often leading to misdiagnosis. Developing technologies for automatically detecting abnormal hand gestures would assist general medical service practitioners with an early diagnosis and treatment.
Methods:
Based on expert experience, we selected three hand gestures with predetermined features and rules as three independent binary classification tasks for abnormal gesture detection. Images from patients with unilateral radial, ulnar, or median nerve injuries and healthy volunteers were obtained using a smartphone. The landmark coordinates were extracted using Google MediaPipe Hands to calculate the features. The receiver operating characteristic curve was employed for feature selection. We compared the performance of rule-based models with logistic regression, support vector machine and of random forest machine learning models by evaluating the accuracy, sensitivity, and specificity.
Results:
The study included 1,344 images, twenty-two patients, and thirty-four volunteers. In rule-based models, eight features were finally selected. The accuracy, sensitivity, and specificity were (1) 98.2, 91.7, and 99.0% for radial nerve injury detection; (2) 97.3, 83.3, and 99.0% for ulnar nerve injury detection; and (3) 96.4, 87.5, and 97.1% for median nerve injury detection, respectively. All machine learning models had accuracy above 95% and sensitivity ranging from 37.5 to 100%.
Conclusion:
Our study provides a helpful tool for detecting abnormal gestures in radial, ulnar, or median nerve injuries with satisfying accuracy, sensitivity, and specificity. It confirms that hand pose estimation could automatically analyze and detect the abnormalities from images of these patients. It has the potential to be a simple and convenient screening method for primary healthcare and telemedicine application.
