Federated Learning Via Inexact ADMM

Area of Science:

• Machine Learning
• Distributed Computing
• Optimization Theory

Background:

• Federated learning (FL) faces challenges in developing efficient optimization algorithms.
• Current FL optimization methods often require full device participation and strict convergence assumptions.
• Gradient descent-based algorithms are widely used but have limitations.

Purpose of the Study:

• To develop a novel, efficient optimization algorithm for federated learning.
• To address the limitations of existing methods, including full device participation and strong convergence assumptions.
• To propose an algorithm that is both computation- and communication-efficient.

Main Methods:

• Development of an inexact alternating direction method of multipliers (ADMM) for federated learning.
• The proposed ADMM method is designed to be robust to stragglers (slow or non-participating devices).
• Analysis of convergence properties under mild conditions.

Main Results:

• The proposed inexact ADMM is computation- and communication-efficient.
• The algorithm effectively combats the stragglers' effect in federated learning.
• Demonstrated high numerical performance compared to state-of-the-art federated learning algorithms.

Conclusions:

• The inexact ADMM offers a superior approach to federated learning optimization.
• This method enhances efficiency and robustness in distributed machine learning settings.
• The algorithm converges under mild conditions, making it broadly applicable.