Exploring personalization via federated representation Learning on non-IID data.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Distributed Systems

Background:

• Federated Learning (FL) enables collaborative model training on decentralized data.
• Statistical heterogeneity in client data distributions poses a significant challenge, leading to global model divergence.
• Existing FL methods struggle with inconsistent data, causing imbalanced features and biased classifiers.

Purpose of the Study:

• To propose a novel two-stage personalized Federated Learning framework, Fed-RepPer.
• To address the issue of statistical heterogeneity in FL by separating representation learning and classification.
• To enhance model performance and personalization on edge devices with limited resources.

Main Methods:

• Implemented a two-stage framework: Fed-RepPer, separating representation learning and classification.
• Utilized supervised contrastive loss for learning robust client-side feature representations.
• Aggregated local representations into a global model, followed by personalized classifier training per client.

Main Results:

• Fed-RepPer demonstrated superior performance compared to alternative methods on heterogeneous datasets (CIFAR-10/100, CINIC-10).
• The framework effectively handles non-IID data by leveraging flexibility and personalization.
• Achieved consistent local objectives and robust representations despite varying data distributions.

Conclusions:

• The proposed Fed-RepPer framework effectively mitigates challenges posed by statistical heterogeneity in FL.
• Separating representation learning and classification enhances model robustness and enables effective personalization.
• Fed-RepPer is suitable for lightweight edge computing environments with resource constraints.