In Vivo Demonstration of Deep Learning-Based Photoacoustic Visual Servoing System.

Area of Science:

• Medical Imaging
• Robotics
• Artificial Intelligence

Background:

• Real-time tracking of surgical tools is crucial for minimally invasive procedures.
• Existing methods often lack precision or speed for complex interventions.
• Photoacoustic imaging offers unique capabilities for visualizing subsurface structures.

Purpose of the Study:

• To develop the first deep learning-based photoacoustic visual servoing system.
• To achieve real-time 3D tracking of catheter tips.
• To integrate point source localization with hybrid position-force control.

Main Methods:

• Developed a novel system combining object detection or instance segmentation with hybrid position-force control.
• Tracked cardiac catheter tips in a phantom and in vivo in swine.
• Evaluated system performance across multiple visual servoing trials.

Main Results:

• Achieved catheter tip detection rates of 86.4-100.0% in vivo.
• Demonstrated tracking errors as low as 0.5 mm in phantom and 0.8 mm in vivo.
• Object detection offered faster inference times (≥145.3 ms) compared to instance segmentation (≥516.3 ms).
• Hybrid position-force control maintained imaging surface contact for ≥99.43% of trials.

Conclusions:

• Successfully demonstrated a deep learning-based photoacoustic visual servoing system.
• Object detection is suitable for real-time applications, while instance segmentation offers lower tracking errors.
• The system shows potential for real-time tracking of various surgical tool tips.