Annotation-efficient medical image segmentation via cross-latent graphs and vector-quantized memory

Medical image segmentation plays a vital role in computer-assisted diagnosis, yet the heavy reliance on large-scale pixel-level annotations limits its scalability in real-world clinical applications. To alleviate this bottleneck, we propose a framework for annotation-efficient medical segmentation that leverages sparse supervision from scribbles and points. The framework is guided by three key principles. First, an auxiliary reconstruction branch is employed to enhance supervision and enrich feature representations derived from limited annotations. Second, a vector quantization (VQ) memory bank stores texture-specific and global features, which are dynamically refined to generate reliable pseudo labels. Third, a cross-latent graph neural network (GNN) exploits non-local dependencies from reconstruction features and transfers this relational knowledge to segmentation features, enabling context-aware and structurally consistent predictions. Extensive experiments on three benchmark datasets (ACDC, BraTS'19, and Pancreas-CT) demonstrate that our framework achieves competitive or superior performance compared with state-of-the-art weakly supervised methods, while approaching fully supervised accuracy. These results highlight the potential of our approach to substantially reduce annotation costs without compromising segmentation quality.