FedSAM-3D: Adapter-Constrained Federated Adaptation for Transferable Medical Segmentation Foundation Models

Transferring large-scale medical foundation models to specific clinical tasks remains challenging, particularly in multi-center scenarios with heterogeneous data distributions and privacy constraints. Existing adaptation strategies provide limited solutions for collaboratively adapting foundation models across institutions while preserving their transferable representations.

We propose FedSAM-3D, a foundation model adaptation framework for multi-center medical image segmentation. Built upon the SAM-Med3D backbone, FedSAM-3D defines the collaborative optimization space within adapter parameters while keeping the pretrained backbone unchanged. Through federated optimization within this constrained adaptation space, our framework enables efficient cross-center knowledge aggregation without exchanging full model parameters, while allowing each client to adapt the foundation model to local medical data distributions.

FedSAM-3D was evaluated on multi-center abdominal organ and brain tumor segmentation datasets under federated adaptation and zero-shot evaluation settings. Across both tasks and multiple clinical datasets, FedSAM-3D generally outperformed ablation variants and existing segmentation methods, demonstrating improved adaptation performance and robustness across heterogeneous medical data distributions. Moreover, FedSAM-3D achieved improved generalization on unseen external datasets, including cross-modality evaluation, highlighting its ability to enhance the transferability of medical foundation models.

FedSAM-3D provides an effective paradigm for federated transfer of medical foundation models, achieving improved adaptation performance and generalization while avoiding direct sharing of raw medical data across institutions.

FedSAM-3D provides a parameter-efficient approach for transferring medical foundation models across institutions without directly sharing raw data, facilitating their potential deployment in diverse clinical environments. Our code is available at https://github.com/huavhuahua/FedSAM-3D.