Factor-Assisted Federated Learning for Personalized Optimization with Heterogeneous Data

Area of Science:

• Machine Learning
• Artificial Intelligence
• Data Science

Background:

• Federated learning (FL) is a privacy-preserving distributed machine learning approach.
• Data heterogeneity across clients is a significant challenge in FL, impacting model performance.
• Existing FL methods struggle with diverse datasets, leading to slower convergence and reduced accuracy.

Purpose of the Study:

• To develop a novel personalized federated learning framework, FedSplit, to address data heterogeneity.
• To improve the convergence rate and prediction performance of deep neural networks in FL.
• To investigate the theoretical and empirical benefits of decomposing client data into shared and personalized components.

Main Methods:

• Proposed FedSplit framework, which decomposes neural network layers into shared and personalized components.
• Developed a novel objective function optimized for this decomposition.
• Introduced factor analysis for practical implementation (FedFac) to decouple hidden elements in real-world datasets.

Main Results:

• FedSplit demonstrates theoretically and empirically faster convergence compared to standard FL methods.
• The generalization bound of FedSplit was analyzed.
• FedFac, the practical implementation, showed superior prediction performance against state-of-the-art methods on real datasets.

Conclusions:

• The FedSplit framework effectively handles data heterogeneity in federated learning.
• Decomposing data into shared and personalized knowledge improves model efficiency and accuracy.
• FedFac provides a practical and high-performing solution for personalized federated learning.