Toward Intraoperative Visual Intelligence: Real-Time Surgical Instrument Segmentation for Enhanced Surgical...

Area of Science:

• Computer-assisted surgery
• Medical imaging analysis
• Surgical robotics

Background:

• Open surgery demands high surgeon visual acuity and spatial awareness for instrument tracking in complex environments.
• Current methods for instrument tracking can be limited by the dynamic and cluttered nature of the surgical field.

Purpose of the Study:

• To develop and validate a real-time system for surgical instrument segmentation and tracking.
• To improve surgeon situational awareness and efficiency in the operating room.
• To explore data-driven insights for enhanced surgical practice.

Main Methods:

• Utilized a head-mounted depth camera for capturing surgical scenes with image and depth data.
• Employed the enhanced U-Net with GridMask (EUGNet) for accurate surgical tool segmentation.
• Implemented Real-Time Streaming Protocol (RTSP) for reliable, low-latency wireless video transmission to a workstation.

Main Results:

• The system achieved 85.5% accuracy in identifying and segmenting surgical instruments in a simulated environment.
• Wireless video transmission demonstrated reliable performance with a latency of 200 ms, suitable for real-time applications.
• The system showed potential for improving situational awareness and surgical efficiency.

Conclusions:

• The developed system represents a significant advancement in assistive surgical technologies.
• Findings suggest a promising pathway towards enhancing surgical practice through AI-driven tools.
• Further development could lead to widespread adoption of such technologies in operating rooms.