Addressing Client Drift in Federated Learning via Class-Prototype Similarity Distillation and Adaptive Mask.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Distributed Systems

Background:

• Federated learning (FL) facilitates collaborative model training across distributed clients while preserving data privacy.
• Non-independent and identically distributed (non-IID) data across clients causes client drift, degrading FL model performance.
• Client drift is primarily driven by increasing logit differences between local and global models due to catastrophic forgetting.

Purpose of the Study:

• To address the performance degradation in federated learning caused by non-IID data.
• To propose a novel algorithm, FedCSD, that aligns local and global model logits.
• To enhance the reliability of knowledge transfer in federated learning frameworks.

Main Methods:

• FedCSD (Federated Class-prototype Similarity Distillation) is introduced to align local and global model logits.
• Leverages similarity between local logits and global prototypes to refine global logits and enhance class similarity information.
• Employs an adaptive mask to filter unreliable global model soft labels, preventing local model optimization errors.

Main Results:

• FedCSD effectively mitigates client drift by aligning local and global model logits.
• The proposed method enhances the quality of class similarity information in the global model.
• Experimental results demonstrate FedCSD's superiority over state-of-the-art FL methods in non-IID scenarios.

Conclusions:

• FedCSD offers a robust solution to the challenge of non-IID data in federated learning.
• The algorithm improves FL performance by addressing catastrophic forgetting and ensuring reliable knowledge distillation.
• The findings highlight the importance of aligning model logits for effective collaborative learning in heterogeneous environments.