SurgMamba: A Hierarchical Hybrid CNN-Mamba Network with Adaptive Fusion for 2D Surgical Image Semantic Segmentation

Area of Science:

• Computer Vision
• Medical Image Analysis
• Artificial Intelligence in Surgery

Background:

• Surgical full scene segmentation is crucial for laparoscopic assistance but faces challenges like visual similarity, illumination variations, and segmenting thin, oblique instruments.
• Existing Mamba-based methods struggle with oblique instrument distributions due to their 2D selective scan strategy.

Purpose of the Study:

• To develop an advanced segmentation method that overcomes limitations in current surgical scene analysis.
• To improve the accuracy of segmenting challenging surgical instruments and anatomical structures in laparoscopic videos.

Main Methods:

• Proposed a holistic vision Mamba block (HVMamba) with a Holistic Directional Selective Scan (HSSD) module to integrate multi-directional spatial features and cross-channel dependencies.
• Developed SurgMamba, a hierarchical hybrid network combining HVMamba and CNN branches for global and local representations, fused adaptively.
• HSSD includes Attention-Guided Holistic Directional Selective Scan (AHSD) and Channel-aware Directional Selective Scan (CASD) for enhanced feature integration.

Main Results:

• SurgMamba demonstrated superior performance compared to state-of-the-art methods on two public datasets.
• Achieved significant improvements in segmenting obliquely oriented thin instruments, specular highlights, and low-contrast boundaries.
• The proposed HVMamba block effectively captures anisotropic spatial features and cross-channel dependencies.

Conclusions:

• SurgMamba offers a robust solution for complex surgical scene segmentation, outperforming existing approaches.
• The novel HVMamba block and network architecture are effective for handling challenging visual conditions in laparoscopic surgery.
• This work advances the potential for AI-driven surgical assistance systems.