Self-supervised learning via cluster distance prediction for operating room context awareness.

Area of Science:

• Computer Vision
• Surgical Robotics
• Machine Learning

Background:

• Intelligent surgical systems require semantic segmentation and activity classification for workflow assistance.
• Current supervised methods for these tasks are not scalable due to extensive annotation needs.
• Self-supervised learning offers a promising alternative to reduce data requirements.

Purpose of the Study:

• To develop a novel 3D self-supervised task for operating room (OR) scene understanding.
• To leverage depth maps from Time-of-Flight (ToF) cameras for improved spatial context learning.
• To generate discriminative features for downstream tasks like semantic segmentation and activity classification.

Main Methods:

• Proposed a new 3D self-supervised learning task focused on predicting relative 3D distances from depth maps.
• Utilized OR scene images captured with ToF cameras.
• Exploited depth information to learn 3D spatial context, moving beyond traditional 2D feature-based pretext tasks.

Main Results:

• The approach was evaluated on two downstream tasks using clinical scenario datasets.
• Demonstrated significant performance improvements, particularly in low-data regimes.
• The self-supervised method proved highly effective in scenarios with limited annotated data.

Conclusions:

• Introduced a novel, privacy-preserving, self-supervised method for OR scene understanding using depth maps.
• Achieved performance comparable to existing self-supervised approaches.
• This method offers a viable solution to mitigate the challenges of full supervision in surgical AI.