Real-Time Tool Localization for Laparoscopic Surgery Using Convolutional Neural Network

Area of Science:

• Medical Robotics
• Computer Vision
• Surgical Technology

Background:

• Partially automated robotic systems are crucial for improving surgical precision and efficiency.
• Laparoscopic surgery requires precise instrument control, often demanding significant manual dexterity.

Purpose of the Study:

• To develop a real-time tool localization method for laparoscopic surgery using convolutional neural networks.
• To enable simultaneous localization of up to two surgical tools.

Main Methods:

• A convolutional neural network model featuring two sequential Hourglass modules was designed.
• The model was trained and evaluated on three datasets: ITAP, Atlas Dione, and EndoVis Challenge.
• Grad-CAM technique was employed for model interpretability.

Main Results:

• The best Hourglass-based model achieved 92.86% accuracy and 27.64 FPS, suitable for robotic integration.
• An independent test set evaluation showed slightly reduced accuracy, suggesting limited generalizability.
• The model demonstrated functional insights via Grad-CAM analysis.

Conclusions:

• The proposed model offers a promising approach for automating laparoscopic surgery tasks.
• Real-time tool localization can enhance surgical efficiency by reducing manual endoscope manipulation.
• Further development is needed to improve generalizability for broader clinical application.