Global prototype distillation for heterogeneous federated learning.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Distributed Systems

Background:

• Federated learning enables collaborative model training while preserving data privacy on local clients.
• Heterogeneous data distributions across clients pose a significant challenge to federated learning systems, degrading global model quality.

Purpose of the Study:

• To address the challenges of data heterogeneity in federated learning.
• To improve the performance of the global model in federated learning systems.

Main Methods:

• Propose global prototype distillation (FedGPD) for heterogeneous federated learning.
• Utilize global class prototypes to guide local training on client devices.
• Align local training objectives with global optima.

Main Results:

• FedGPD demonstrates improved performance in heterogeneous federated learning scenarios.
• Experiments show FedGPD outperforms state-of-the-art methods.
• Achieved average accuracy improvements of 0.22% to 1.28% on benchmark datasets.

Conclusions:

• Global prototype distillation is an effective strategy for enhancing federated learning on heterogeneous data.
• FedGPD successfully improves global model quality by leveraging distilled knowledge from global prototypes.
• The proposed method offers a promising solution for practical federated learning applications with data diversity.