Weakly supervised convolutional LSTM approach for tool tracking in laparoscopic videos.

Area of Science:

• Computer Vision
• Medical Imaging
• Surgical Robotics

Background:

• Real-time surgical tool tracking is crucial for intelligent operating rooms.
• Current methods require extensive manual annotation of tool positions, which is time-consuming and difficult.
• There is a need for more efficient methods for training surgical tool trackers.

Purpose of the Study:

• To propose a novel approach for surgical tool tracking using only binary presence annotations.
• To train a deep learning model for laparoscopic surgical videos without manual spatial annotations.
• To demonstrate the sufficiency of binary presence labels for training an effective tool tracker.

Main Methods:

• A Convolutional Neural Network (CNN) combined with a Convolutional Long Short-Term Memory (ConvLSTM) neural network was developed.
• The model was trained end-to-end using only binary presence labels (weak supervision).
• ConvLSTM was utilized to capture temporal dependencies and spatio-temporal information for smoothing localization heat maps.

Main Results:

• The proposed ConvLSTM-based tracker significantly outperformed a baseline CNN tracker.
• Improvements were observed in tool presence detection, spatial localization, and motion tracking.
• Quantitative improvements exceeded [Formula: see text], [Formula: see text], and [Formula: see text] for the respective metrics.

Conclusions:

• Binary presence labels are sufficient for training deep learning-based surgical tool tracking models.
• The ConvLSTM effectively utilizes spatio-temporal coherence in video frames.
• The proposed method enhances tool presence detection, spatial localization, and motion tracking accuracy.