Enhancing Trustworthiness of Semantic Segmentation in Cataract Surgery Videos via Intra-Phase Label Propagation

Area of Science:

• Ophthalmology
• Computer Vision
• Medical Imaging

Background:

• Accurate semantic segmentation of instruments is crucial for cataract surgery assistance and skill assessment.
• Existing methods struggle with instance-level feature similarity across surgical phases, leading to unreliable instrument categorization.
• Blurred edges in surgical videos pose a challenge for precise object delineation.

Purpose of the Study:

• To develop a robust label propagation framework for accurate semantic segmentation of cataract surgery videos.
• To leverage phase-specific instrument consistency for improved segmentation performance.
• To address challenges like blurred edges and ensure real-time applicability.

Main Methods:

• A label propagation framework utilizing initial frame labels to predict masks for subsequent frames.
• A pseudo-label generation and filtering strategy for obtaining reliable initial frame labels.
• A fixed-size memory bank with an adaptive update module and a semantic edge perception module.

Main Results:

• Achieved 80.7% mIoU on a 14-category public dataset and 88.8% mIoU on a 12-category private dataset.
• Significantly outperformed state-of-the-art and other label propagation-based methods.
• Demonstrated minimized memory consumption and maintained ~30 FPS processing speed.

Conclusions:

• The proposed label propagation framework enables precise and trustworthy semantic segmentation of cataract surgery videos.
• The method effectively handles instance-level feature similarity and blurred edges, improving surgical video analysis.
• This approach offers a computationally efficient and accurate solution for real-world surgical applications.