Related Experiment Video
Updated: Dec 28, 2025
Capturing Dynamic Finger Gesturing with High-resolution Surface Electromyography and Computer Vision
Published on: March 28, 2025
Enhancement of surgical hand gesture recognition using a capsule network for a contactless interface in the operating
A-Reum Lee1, Yongwon Cho1, Seongho Jin1
1Department of Convergence Medicine, University of Ulsan College of Medicine, Asan Medical Center, Seoul, South Korea.
Background And Objective:
Hand gesture recognition systems in operating rooms (ORs) are crucial for browsing and controlling computer-aided devices, which have been developed to decrease the risk of contamination during surgical procedures.
Methods:
We proposed the use of hand gesture recognition to enhance accuracies and recognition areas with the capsule network (CapsNet) of deep neural network and Leap Motionâ Our method includes the i) extraction and preprocessing of infrared (IR) images (60 frames per second) from Leap Motion™, ii) training of various types of networks, and iii) gesture recognition evaluation in the OR. We trained the images of training dataset (N=903) and tested images (N=100) using five types of surgical hand gestures including hovering, grab, click, one peak, and two peaks by 10 subjects with various types of augmentation methods including rotate (0∘, 90∘, 180∘), scale, translation, illumination, and resize.
Results:
CapsNet achieved a classification accuracy of 86.46% (around 10% improvement) compared with 73.67% for the baseline convolutional neural network (CNN) and 76.4% for VGG16.
Conclusions:
In conclusion, the accuracy of hand gesture recognition with CapsNet was better than that of conventional CNNs, which could be used to navigate and manipulate various types of computer-aided devices and applications through contactless gesture interaction.
