DFedADMM: Dual Constraint Controlled Model Inconsistency for Decentralize Federated Learning.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Distributed Systems

Background:

• Federated Learning (FL) faces communication burdens.
• Decentralized Federated Learning (DFL) removes central servers but struggles with local inconsistency and heterogeneous overfitting.

Purpose of the Study:

• To propose novel DFL algorithms, DFedADMM and DFedADMM-SAM, to enhance DFL performance.
• To address challenges of local inconsistency and local heterogeneous overfitting in DFL.

Main Methods:

• DFedADMM utilizes primal-dual optimization (ADMM) with dual variables to manage model inconsistency from heterogeneous data.
• DFedADMM-SAM incorporates Sharpness-Aware Minimization (SAM) to mitigate overfitting by generating locally flat models.

Main Results:

• Theoretical convergence rates derived for both DFedADMM and DFedADMM-SAM in non-convex settings.
• Empirical validation on MNIST, CIFAR10, and CIFAR100 datasets shows superior performance.

Conclusions:

• DFedADMM and DFedADMM-SAM significantly improve generalization, convergence speed, and reduce communication overhead in DFL.
• The proposed methods offer effective solutions to persistent challenges in decentralized machine learning.