Federated lung nodule segmentation using a hybrid transformer-U-Net architecture

Early and accurate identification of malignant pulmonary nodules is essential to reduce lung cancer mortality, yet segmentation of these nodules in CT scans remains challenging due to their small size, variable morphology, low contrast, and extreme class imbalance. To address these challenges, we propose a privacy-preserving federated learning framework that integrates a hybrid U-Net-Transformer architecture with a customized Dice-Focal loss function. The hybrid model combines residual convolutional blocks for local texture feature extraction with Transformer-based self-attention for capturing global contextual dependencies, while skip connections, Layer Normalization, and Dropout enhance gradient flow and generalization across heterogeneous client datasets. The Dice-Focal loss mitigates class imbalance and improves boundary sensitivity, yielding stable convergence and precise segmentation of solid nodules. Using the LUNA16 dataset, our federated setup demonstrated high segmentation fidelity for nodules between 15mm and 25mm, achieving Dice coefficients up to 0.93 and robust precision-recall trade-offs across clients. These results highlight the framework's potential for scalable, privacy-preserving lung cancer screening, while also identifying challenges in segmenting part-solid or low-contrast nodules for future methodological improvements.