Related Experiment Video
Updated: Jun 11, 2026
05:12
Robotized Testing of Camera Positions to Determine Ideal Configuration for Stereo 3D Visualization of Open-Heart Surgery
Published on: August 12, 2021
Streamlining stereo-differentiable rendering for marker-free real-time tracking of surgical robots
Yanghe Hao1, Martin Huber1, Christos Bergeles1
1School of Biomedical Engineering and Imaging Sciences, King's College London, London, UK.
Summary
This study introduces marker-free surgical robot tracking using stereo-differentiable rendering, achieving real-time performance and high accuracy. This approach enhances safety and efficiency in surgical environments.
Area of Science:
- Robotics
- Computer Vision
- Surgical Technology
Background:
- Marker-based tracking in surgical robotics is prone to occlusion in cluttered environments.
- Current methods may require extensive setup times and limit multi-robot interactions.
Purpose of the Study:
- To evaluate stereo-differentiable rendering for marker-free, real-time surgical robot pose estimation.
- To mitigate limitations of marker-based tracking, improving safety and enabling advanced robotic capabilities.
Main Methods:
- Extended the roboreg framework with sequential optimization and motion-adaptive hyperparameter tuning for dynamic tracking.
- Integrated CUDA stream parallelization and CUDA-graph acceleration for enhanced segmentation and optimization speed.
- Evaluated performance using diverse datasets, including occluded scenarios, comparing against marker-based and other marker-free methods.
Main Results:
- Achieved real-time localization at 30 fps for 1080p video, a significant speedup from 14 fps.
- Demonstrated high accuracy with near 1 cm translational and sub-degree rotational error.
- Outperformed FoundationPose in dynamic and static estimations, achieving 6x faster inference.
Conclusions:
- Stereo-differentiable rendering enables real-time, high-resolution, marker-free tracking of surgical robots.
- The developed method achieves accuracy comparable to marker-based systems and surpasses existing marker-free baselines.
