Dynamic Client Distillation for Semi-Supervised Federated Learning in a Realistic Scenario

Area of Science:

• Artificial Intelligence
• Machine Learning
• Medical Informatics

Background:

• Semi-supervised federated learning (SSFL) advances public health by enabling medical data sharing.
• Existing SSFL methods assume uniform data labeling (labels-at-server/client), which is unrealistic for diverse medical institutions.

Purpose of the Study:

• To develop a novel SSFL framework (FedCD) for realistic client data scenarios with varying annotation levels (fully-labeled, partially-labeled, fully-unlabeled).
• To maximize the utility of unlabeled data within client federations and adapt to heterogeneous data distributions.

Main Methods:

• Proposed FedCD framework with three client-distilled models for distinct data distributions.
• Employed server-client federation and knowledge distillation for parameter condensation.
• Implemented dynamic adjustment of client model contributions based on proximity to distilled models.
• Utilized aggregated client-distilled models for model drift correction.

Main Results:

• FedCD effectively harnesses unlabeled data and accommodates diverse annotation levels.
• The framework adapts to varying data distributions and mitigates parameter drift in heterogeneous models.
• Demonstrated superiority on two medical image segmentation and one classification task.

Conclusions:

• FedCD addresses realistic challenges in medical data scenarios by integrating diverse annotation levels and data distributions.
• The dynamic federated approach enhances the efficiency and adaptability of SSFL in healthcare applications.