Recurrent multi-view 6DoF pose estimation for marker-less surgical tool tracking.

Area of Science:

• Computer Vision
• Medical Technology
• Machine Learning

Background:

• Marker-based tracking in surgical navigation is precise but requires extensive preparation and is susceptible to marker occlusion.
• Deep learning offers a promising marker-less alternative using RGB videos for surgical instrument tracking.

Purpose of the Study:

• To apply object pose estimation with a novel deep learning architecture for marker-less surgical instrument tracking.
• To address challenges of time-consuming preparation and marker occlusion in surgical navigation.

Main Methods:

• Combined multi-view pose estimation with recurrent neural networks (RNNs) to leverage temporal coherence.
• Integrated a spatio-temporal feature extractor into an existing pose estimation pipeline for sequence-based feature incorporation.
• Evaluated performance under conditions of instrument occlusion.

Main Results:

• Achieved mean tip error below 1.0 mm and angle error below 0.2° on a synthetic dataset with a four-camera setup.
• Attained an error below 3.0 mm on a real dataset using four cameras.
• The recurrent approach demonstrated ~3 mm greater precision in tip position prediction during limited instrument visibility compared to non-recurrent methods.

Conclusions:

• Deep learning-based tracking with multiple cameras shows competitiveness with marker-based systems for surgical instruments.
• Recurrent temporal information significantly benefits tracking reliability when instruments are occluded.
• The combination of multi-view processing and recurrent networks enhances the precision and usability of surgical pose estimation.