Federated Learning via Variational Bayesian Inference: Personalization, Sparsity and Clustering.

Area of Science:

• Machine Learning
• Artificial Intelligence
• Data Science

Background:

• Federated learning (FL) enables distributed machine learning while preserving client privacy.
• FL performance is often degraded by heterogeneous and limited client data.
• Existing personalized FL methods struggle with extreme data heterogeneity.

Purpose of the Study:

• To introduce novel personalized Bayesian FL approaches to address performance degradation in FL.
• To enhance inference efficiency and handle extreme data heterogeneity in FL.
• To provide theoretical guarantees and empirical validation for the proposed methods.

Main Methods:

• Developed pFedBayes: personalized Bayesian FL using global distribution as prior.
• Introduced sFedBayes: a sparse variant for enhanced inference efficiency.
• Proposed cFedbayes: a clustered Bayesian FL model for extreme non-i.i.d. data.

Main Results:

• Theoretical analysis provides generalization error bounds, achieving minimax optimality.
• CFedbayes demonstrates cluster-level generalization error bounds, outperforming uniform bounds.
• Experimental results show superior performance of proposed methods over advanced personalized FL techniques.

Conclusions:

• The proposed Bayesian FL approaches effectively mitigate performance degradation caused by data heterogeneity and limitations.
• pFedBayes, sFedBayes, and cFedbayes offer improved privacy, efficiency, and accuracy in personalized FL.
• These methods represent a significant advancement for practical FL applications with diverse and scarce data.