Training-free temporal object tracking in surgical videos.

Area of Science:

• Medical Image Analysis
• Computer Vision
• Surgical Technology

Background:

• Accurate tracking of anatomical structures and instruments in laparoscopic cholecystectomy (LC) videos is crucial for surgical safety and training.
• Existing methods face challenges due to high costs of pixel-level annotations and data inconsistencies.
• There is a need for efficient and accurate object tracking solutions in surgical video analysis.

Purpose of the Study:

• To present a novel approach for online object tracking in LC surgical videos.
• To target the localisation and tracking of critical anatomical structures and surgical instruments.
• To overcome limitations of costly annotations and label inconsistencies in current datasets.

Main Methods:

• Leveraging pre-trained text-to-image diffusion models for feature extraction without fine-tuning.
• Utilizing extracted features and cross-frame interactions via an affinity matrix for temporal continuity.
• Employing a query-key-value attention-inspired mechanism for robust tracking.

Main Results:

• Diffusion features demonstrate superior object localisation and semantic consistency across frames and decoder levels.
• The proposed tracking framework outperforms existing methods in temporal object tracking.
• Achieved $79.19\%$ per-pixel accuracy, $56.20\%$ Jaccard score, and $79.48\%$ F-score on the CholeSeg8K dataset.

Conclusions:

• Introduces a novel application of text-to-image diffusion models in surgical video analysis.
• Offers a cost-effective and accurate solution for temporal object tracking in minimally invasive surgery.
• Advances the field of surgical video analysis with improved tracking capabilities.