Cascade Multi-Level Transformer Network for Surgical Workflow Analysis.

Area of Science:

• Computer Vision
• Medical Image Analysis
• Artificial Intelligence in Surgery

Background:

• Surgical workflow analysis is crucial for context-aware computer-aided surgical systems.
• Existing deep learning methods often use single-level temporal context aggregation, neglecting frame-specific multi-level information needs.
• Accurate surgical phase prediction requires understanding context at multiple granularities.

Purpose of the Study:

• To propose a novel deep learning network, the Cascade Multi-Level Transformer Network (CMTNet), for improved surgical phase recognition.
• To address the limitations of single-level temporal context aggregation in existing methods.
• To enhance the adaptiveness and accuracy of surgical phase prediction from video data.

Main Methods:

• Developed CMTNet, featuring cascaded Adaptive Multi-Level Context Aggregation (AMCA) modules.
• Each AMCA module adaptively fuses frame-specific spatial features with frame-level and phase-level temporal contexts.
• Introduced a novel refinement loss to guide AMCA modules for improved prediction confidence and smoothness.

Main Results:

• CMTNet progressively enriches frame representations with required multi-level semantics for frame-adaptive prediction.
• The proposed refinement loss effectively enhances context extraction quality.
• Achieved state-of-the-art performance on the Cholec80 and M2CAI datasets.

Conclusions:

• CMTNet demonstrates superior performance in surgical phase recognition compared to existing methods.
• The adaptive multi-level context aggregation and refinement loss are key to CMTNet's effectiveness.
• The proposed approach advances the field of computer-aided surgical systems through enhanced video analysis.