A touch-free human-robot collaborative surgical navigation robotic system based on hand gesture recognition

Area of Science:

• Robotics
• Surgical Technology
• Human-Robot Interaction

Background:

• Robot-assisted minimally invasive surgery (RAMIS) is increasingly adopted.
• Current touch-based systems pose infection risks due to manual equipment handling and sterilization needs.
• Precise, touch-free robotic manipulation is a significant challenge.

Purpose of the Study:

• To develop a novel human-robot interaction (HRI) interface for contactless control of surgical robots.
• To enable precise manipulation of surgical instruments using hand gesture recognition.
• To reduce bacterial diffusion risks associated with traditional surgical robotics.

Main Methods:

• Developed a gesture recognition interface using hand-keypoint regression and hand-shape reconstruction.
• Encoded 21 keypoints from recognized hand gestures to control robot actions.
• Evaluated system accuracy through phantom and cadaver studies.

Main Results:

• Phantom study: average needle tip location error of 0.51 mm and mean angle error of 0.34 degrees.
• Cadaver study (simulated biopsy): needle insertion error of 0.16 mm and angle error of 0.10 degrees.
• Demonstrated clinically acceptable accuracy for contactless surgical tasks.

Conclusions:

• The proposed gesture-based HRI system enables precise, contactless control of surgical robots.
• This technology significantly reduces infection risks in robot-assisted surgery.
• The system shows high accuracy and applicability for assisting surgeons in minimally invasive procedures.