FPGA-Based Low-Latency Semantic Feedback System for Real-Time Instrument Localization in Telesurgery

Area of Science:

• Robotics
• Medical Engineering
• Computer Vision

Background:

• Telesurgery is significantly hindered by visual latency, which impairs critical master-slave synchronization and overall patient safety.
• Existing telesurgical systems struggle to overcome the challenges posed by visual delays, impacting precision and reliability.

Purpose of the Study:

• To develop and evaluate a real-time semantic feedback framework designed to minimize visual latency in telesurgery.
• To maintain millimetre-level localization accuracy for surgical instruments despite reduced latency.

Main Methods:

• A low-latency semantic feedback system was engineered, utilizing a custom visual marker and FPGA-based extraction for real-time instrument localization.
• System validation involved phantom experiments, accuracy benchmarking against the NDI system, precise latency measurements, and task-based evaluations under simulated visual delay.

Main Results:

• The system demonstrated a high marker extraction success rate of 99.2%.
• Exceptional localization accuracy was achieved, with a maximum mean error of 0.68 ± 0.72 mm.
• The end-to-end system latency was measured at a low 31.2 ± 1.4 ms, and task completion times were significantly reduced under delayed visual feedback.

Conclusions:

• The developed semantic feedback system offers robust, precise, and low-latency performance.
• This technology shows strong potential for mitigating the adverse effects of visual latency on surgical performance in telesurgery.
• Further in vivo validation is recommended to support its integration into advanced telesurgical systems.