Multi-granularity cross-image semantic modeling for medical ultrasound image segmentation

Automatic segmentation of medical ultrasound images is critical for computer-aided diagnosis (CAD). However, most existing methods primarily exploit within-image semantics while neglecting cross-image semantic modeling. By leveraging cohort-level shape and echotexture priors, cross-image semantic modeling improves robustness against inter-patient and inter-scanner variations. Nevertheless, positional correlations in cross-image scenarios may incorrectly map foreground features from previous images onto background regions of the current one, causing semantic conflicts that degrade segmentation reliability. Moreover, many methods emphasize global context while overlooking fine-grained lesion information. To address these limitations, we propose Multi-granularity Cross-image Semantic Modeling (MCSM), where multi-granularity denotes the joint modeling of global (whole-image) and local (lesion-centric local patch) representations. Specifically, we first design a Multi-Granularity Patch Extraction Process (MGPEP) that extracts lesion patches and fuses them with whole-image features to strengthen cross-image semantics. We then propose a Frequency-domain Feature Dependency Module (FFDM) that performs multi-granularity spectral filtering to capture cross-image dependencies. During training, features from preceding batches (i.e., previous images) are mapped to the frequency domain to extract latent semantic correlations and are integrated into current batches (i.e., current image) representation, thereby enhancing foreground-background discrimination. In addition, we employ dual-mean filtering to suppress high-frequency noise in the global spectrum. Extensive experiments demonstrate the effectiveness and robustness of MCSM over state-of-the-art methods, highlighting its potential for clinical application. Code and models are available at https://github.com/XyL215/MCSM.