3D UXFormer: Dual-Branch Feature Fusion for Precise Breast MRI Multi-Tissue Segmentation

Accurate segmentation of breast tissues in MRI is essential for precise breast cancer diagnosis and effective treatment planning. Manually segmenting these tissues is not only arduous but also time-consuming. Fusing multi-phase MRI images, like the Contrast-enhanced Early ([Formula: see text]) and Contrast-enhanced Peak ([Formula: see text]) phases, to fully utilize their complementary information remains a challenge. In this paper, we introduce a novel 3D UXFormer model. It combines the 3D UNET and Vision Transformer (ViT) through a dual-branch structure, capitalizing on the 3D UNET's local feature extraction prowess and ViT's global feature-capturing ability. To address data-related issues, we pre-train ViT using the Masked Autoencoder (MAE) approach. Extensive experiments demonstrate that the 3D UXFormer outperforms state-of-the-art algorithms, achieving higher Dice and IoU scores across diverse breast tissues in different datasets. The source code of our method is available on Github.