FedADMM-InSa: An inexact and self-adaptive ADMM for federated learning

Area of Science:

• Artificial Intelligence
• Machine Learning
• Distributed Systems

Background:

• Federated learning (FL) enables privacy-preserving distributed model training.
• Existing FL methods, including FedADMM, face challenges with heterogeneous data/systems and hyperparameter sensitivity.
• Efficient FL requires addressing communication bottlenecks and local resource constraints.

Purpose of the Study:

• To develop a more robust and efficient federated learning algorithm that minimizes hyperparameter tuning.
• To reduce computational costs and mitigate the straggle effect in federated learning.
• To improve the overall performance and applicability of federated learning across diverse datasets and systems.

Main Methods:

• Proposing FedADMM-InSa, an inexact and self-adaptive federated ADMM algorithm.
• Implementing an inexactness criterion for local client updates, independent of empirical local training accuracy.
• Introducing a self-adaptive scheme for dynamic adjustment of client penalty parameters.

Main Results:

• FedADMM-InSa demonstrates improved model accuracy by 7.8% compared to benchmark algorithms.
• Client local workloads are reduced by 55.7% through the inexactness criterion and adaptive penalty.
• The algorithm shows resilience to data and system heterogeneity without extensive hyperparameter tuning.

Conclusions:

• FedADMM-InSa offers a more efficient and robust solution for federated learning, particularly in heterogeneous environments.
• The proposed inexactness criterion and self-adaptive penalty parameter adjustment significantly reduce computational overhead and improve performance.
• This work advances federated learning by simplifying hyperparameter management and enhancing practical applicability.