VGM-UNet: A hybrid visual graph deformable mamba with fourier neural operator U-Net for medical image segmentation

Deep learning methods have demonstrated remarkable advancements in medical image segmentation. However, achieving high accuracy remains a prominent challenge. In this paper, we present a new architecture, named VGM-UNet, that improves U-shaped segmentation models in performance and expressiveness by introducing the Structured State Space Duality algorithm to combine Graph Neural Networks, sparse attention, and Mamba-2, into U-Net and yield the best of these designs. This is accomplished through three primary modifications: we first adopt a novel approach, constructing a 2D State Space Model and an eight-way multi-scanning module, thereby creating the Vision Mamba-2, which serves as the foundation for building a hierarchical visual backbone that can be directly applied to the graph structure of image patches. Then, based on the Fast Fourier transform, we construct a Feed-Forward Network module, as a complement to Mamba, to model channel contents and improve the accuracy of capturing small objects. Moreover, on the basis of the modular architecture, we build a simple yet powerful U-shaped hybrid network, which simplifies the model design and enhances the model's expressiveness. These changes alleviate the limitations of conventional U-shaped architectures in accuracy improvements and achieve impressive results. We validate VGM-UNet through extensive experiments, demonstrating that our model outperforms existing state-of-the-art models in terms of segmentation accuracy on the Synapse and ACDC benchmark datasets. The experimental results also indicate that the Visual Graph State Space module can be conveniently applied to various medical image segmentation tasks.