Contrastive encoder pre-training-based clustered federated learning for heterogeneous data.

Area of Science:

• Machine Learning
• Artificial Intelligence
• Distributed Systems

Background:

• Federated learning (FL) enables collaborative model training while preserving data privacy.
• Data heterogeneity in FL significantly degrades model performance.
• Clustered federated learning (CFL) aims to create personalized models for client groups.

Purpose of the Study:

• To address the clustering failure issue in CFL caused by a lack of pre-trained models.
• To improve the performance and convergence of FL systems in heterogeneous environments.
• To propose a novel approach, contrastive pre-training-based clustered federated learning (CP-CFL).

Main Methods:

• Utilizing self-supervised contrastive learning for pre-training FL systems with unlabeled data.
• Implementing a client clustering strategy based on local model selection.
• Combining self-supervised pre-training with client clustering to form CP-CFL.

Main Results:

• CP-CFL effectively tackles data heterogeneity issues in FL.
• The proposed method demonstrates improved model convergence.
• Extensive experiments in heterogeneous FL settings validate the effectiveness of CP-CFL.

Conclusions:

• Self-supervised pre-training is crucial for effective client clustering in FL.
• CP-CFL offers a robust solution for improving FL performance under data heterogeneity.
• The study highlights the potential of leveraging unlabeled data in distributed learning environments.