Sparse Personalized Federated Learning.

Area of Science:

• Machine Learning
• Artificial Intelligence
• Distributed Systems

Background:

• Federated learning (FL) enables collaborative model training without accessing private client data.
• Key challenges in FL include statistical diversity, limited client computing power, and high communication overhead.
• Existing personalized FL methods often struggle with efficiency and effectiveness under data heterogeneity.

Purpose of the Study:

• To introduce FedMac, a novel sparse personalized federated learning scheme.
• To address statistical diversity, reduce computational and communication burdens in FL.
• To improve the performance and efficiency of personalized federated learning.

Main Methods:

• FedMac incorporates an approximated $\ell _{1}$-norm and client-model-global-model correlation into the FL loss function.
• This approach promotes sparse personalization by optimizing the correlation between client and global models.
• Convergence analysis and theoretical evaluations were conducted to validate the scheme's effectiveness.

Main Results:

• FedMac significantly improves performance on datasets with statistical diversity.
• The scheme effectively reduces communication and computational loads compared to non-sparse FL methods.
• Experimental results show FedMac outperforms state-of-the-art personalization methods across various datasets (MNIST, FMNIST, CIFAR-100, Synthetic, CINIC-10).

Conclusions:

• FedMac offers a robust solution for sparse personalized federated learning.
• The proposed method enhances model performance and efficiency without compromising convergence.
• FedMac demonstrates superior sparse personalization capabilities compared to $\ell _{2}$-norm based methods.