Toward real-time autonomous navigation: transformer-based catheter tip tracking in fluoroscopy.

Area of Science:

• Medical Imaging
• Robotics
• Artificial Intelligence

Background:

• Mechanical thrombectomy (MT) is crucial for stroke treatment but faces access limitations.
• Reinforcement learning (RL) based robotic systems offer potential for autonomous navigation in MT.
• Accurate real-time catheter tip tracking is essential for current RL methods.

Purpose of the Study:

• Develop and evaluate a real-time catheter tip tracking pipeline under fluoroscopy.
• Address challenges like low contrast, noise, and device occlusion in fluoroscopic images.
• Provide a foundation for RL-based autonomous navigation in MT.

Main Methods:

• Designed a multi-threaded pipeline including frame reading, preprocessing, inference, and post-processing.
• Trained and benchmarked deep learning segmentation models (U-Net, U-Net+Transformer, SegFormer).
• Utilized two-step component filtering, medial skeletonization, and arc-length path following for post-processing.

Main Results:

• The two-class SegFormer model achieved a mean absolute error of 4.44 mm on manually labeled data.
• This outperformed U-Net (4.60 mm), U-Net+Transformer (6.20 mm), and three-class models (5.19-7.74 mm).
• The system surpassed state-of-the-art CathAction results, improving Dice scores by up to +5% for three-segmentation.

Conclusions:

• The proposed tracking framework demonstrates stable performance in challenging imaging conditions.
• It outperforms prior benchmarks, offering a reliable and efficient solution.
• This technology supports the advancement of RL-based autonomous mechanical thrombectomy navigation.