Semantic-Preserving Surgical Video Retrieval With Phase and Behavior Coordinated Hashing

Area of Science:

• Computer Science
• Medical Informatics
• Biomedical Engineering

Background:

• Surgical video retrieval is crucial for education and knowledge transfer.
• Current methods often fail due to a lack of semantic understanding.
• Integrating hierarchical and coordinated semantics presents a significant challenge.

Purpose of the Study:

• To propose a novel semantic-preserving surgical video retrieval (SPSVR) framework.
• To address the limitations of existing retrieval techniques by incorporating surgical phase and behavior semantics.
• To improve the accuracy and relevance of surgical video retrieval.

Main Methods:

• A dual-level hashing module is employed to capture the hierarchical relationship between coarse- and fine-grained semantics.
• Surgical phase and behavior similarities are transformed into high- and low-level similarities within a shared Hamming space.
• A self-supervised learning scheme is utilized for behavior semantics to reduce reliance on fine-grained annotations.

Main Results:

• The proposed SPSVR framework demonstrates robust performance across four surgical video datasets from two disciplines.
• Clinical validation experiments confirm the method's ability to retrieve surgeon-expected results.
• The dual-level hashing module effectively preserves semantics in binary hash codes.

Conclusions:

• The SPSVR framework offers an effective solution for semantic-preserving surgical video retrieval.
• The method enhances the discovery of relevant surgical content for education and knowledge transfer.
• The approach shows promise for improving clinical decision-making and surgical training.