Real world federated learning with a knowledge distilled transformer for cardiac CT imaging

Area of Science:

• Medical Imaging
• Artificial Intelligence
• Federated Learning

Background:

• Federated learning (FL) enables decentralized data analysis while preserving privacy.
• Real-world FL applications struggle with partially labeled datasets, leaving unlabeled data underutilized.
• Leveraging unlabeled data can enhance transformer models, especially with limited and diverse annotations.

Purpose of the Study:

• To develop and evaluate a semi-supervised federated learning strategy for cardiac CT analysis.
• To improve the performance of transformer architectures using partially labeled data across multiple institutions.
• To conduct the largest federated cardiac CT analysis to date.

Main Methods:

• A two-step semi-supervised strategy was employed, distilling knowledge from Convolutional Neural Networks (CNNs) to a transformer architecture.
• CNNs predicted labels on unlabeled data, and the transformer learned from these predictions using label-specific heads.
• The study involved 8,104 cardiac CT scans across eight hospitals in a real-world setting.

Main Results:

• The proposed method improved predictive accuracy by effectively utilizing unlabeled data.
• Simultaneous learning of all partial labels across the federated network was achieved.
• The transformer-based approach demonstrated superior generalizability on downstream tasks compared to UNet-based models.

Conclusions:

• The developed semi-supervised federated learning strategy successfully leverages unlabeled data to enhance transformer performance in cardiac CT analysis.
• This approach addresses the challenge of partially labeled datasets in real-world federated settings.
• Openly available code and model weights aim to facilitate future advancements in cardiac CT analysis.