Enhancing Trustworthiness of Semantic Segmentation in Cataract Surgery Videos via Intra-Phase Label Propagation

Accurate segmentation of semantic features is a pivotal procedure for cataract surgery assistance, surgical skill assessment and related applications. However, previous studies have failed to consider the instance-level feature similarity of instruments across different surgical phases in cataract surgery videos, leading to unreliable decision-making regarding instrument categories. In this study, we propose a label propagation framework to effectively leverage the consistency of phase-specific instruments, which utilizes the initial frame labels from each surgical phase to predict masks for the remaining frames, achieving precise and trustworthy semantic segmentation of cataract surgery videos. Specifically, we design a pseudo-label generation and filtering strategy to automatically obtain highly reliable initial frame labels for each surgical phase. In addition, we establish a fixed-size memory bank with an adaptive update module to ensure long-term applicability in real surgical environments. To address the common problem of blurred edges in cataract surgery scenes, we develop a semantic edge perception module to allow the model to focus on and distinguish the edges of different objects. The proposed method achieved an mIoU of 80.7% and 88.8% on a publicly available dataset (14 categories) and a private dataset (12 categories) with a total of 9,723 frames, respectively, significantly outperforming the state-of-the-art methods and other label propagation-based approaches. Furthermore, our method minimizes memory consumption and maintains about 30 FPS while processing long video sequences.