A non-contact interactive system for multimodal surgical robots based on LeapMotion and visual tags

Area of Science:

• Robotics
• Surgical Technology
• Human-Computer Interaction

Background:

• Robotic integration in minimally invasive surgery is increasing.
• Conventional contact-based controls risk bacterial infection, limiting robotic applications.
• A need exists for sterile, non-contact human-robot interaction methods in surgery.

Purpose of the Study:

• To develop and evaluate a non-contact gesture and visual tag-based control system for surgical robots.
• To enable surgeons to control and fine-tune robotic instruments without physical contact.
• To enhance safety and flexibility in robotic-assisted surgery.

Main Methods:

• Encoding six distinct gestures captured by Leap Motion for robot control.
• Utilizing Aruco visual tags for accurate 3D spatial positioning.
• Developing 12 fine-tuning operations for surgical instrument manipulation.

Main Results:

• The system successfully controlled surgical robots using non-contact gestures.
• Accurate 3D positioning of visual tags enabled precise instrument refinement.
• Experimental evaluation demonstrated sufficient precision for clinical application.

Conclusions:

• The proposed non-contact system meets clinical standards for robotic surgery.
• It offers surgeons a safe and flexible interaction method, reducing infection risks.
• This technology advances the potential of robotics in minimally invasive procedures.