Related Experiment Video
Updated: Dec 17, 2025
10:25
Technical Approach for Infrared Tracking for Soft Tissue Navigation with a Holographic Head-Mounted Display and Preclinical Validation
Published on: September 2, 2025
392
Object extraction via deep learning-based marker-free tracking framework of surgical instruments for
Jiayi Zhang1, Xin Gao2,3
1Suzhou Institute of Biomedical Engineering and Technology, Chinese Academy of Sciences, No. 88 Keling Road, Suzhou New District, Suzhou, 215163, Jiangsu, China.
Summary
This study introduces a novel deep learning framework for marker-free surgical instrument tracking in laparoscopic surgery. The new method achieves 100% tracking success and improves accuracy, aiding robotic surgery advancements.
Area of Science:
- Robotics
- Computer Vision
- Surgical Technology
Background:
- Visual servoing is crucial for active control in laparoscope-holder robots, requiring precise surgical instrument tracking.
- Marker-free tracking frameworks are essential for seamless integration and application in minimally invasive surgery.
- Deep learning (DL) offers advanced capabilities for real-time object recognition and tracking in complex surgical environments.
Purpose of the Study:
- To present a novel marker-free surgical instrument tracking framework utilizing deep learning-based object extraction.
- To enable accurate and real-time tracking of surgical instruments for visual servoing applications in robotic surgery.
- To enhance the performance of laparoscope-holder robots in laparoscopic procedures.
Main Methods:
- A deep learning segmentation model was trained to extract surgical instrument components (end-effector and shaft) in real time.
- Extracted objects were converted into distance images, and central points were identified to determine the tracking point.
- The framework was validated using in vivo laparoscopic videos, comparing its performance against a tracking-by-detection approach.
Main Results:
- The proposed framework achieved a 100% mean tracking success rate in seven in vivo laparoscopic videos.
- Mean tracking accuracy was (3.9 ± 2.4, 4.0 ± 2.5) pixels, with a mean tracking speed of 15 fps.
- The DL-based object extraction method demonstrated a significant improvement in mean tracking accuracy by 37% and 23% compared to a tracking-by-detection method.
Conclusions:
- The developed object extraction via DL-based marker-free tracking framework enables accurate and fast tracking of surgical instruments in laparoscopic videos.
- This advancement holds significant guiding importance for the practical application of laparoscope-holder robots in laparoscopic surgeries.
- The framework provides a robust solution for real-time instrument localization, crucial for enhanced robotic surgical performance.
