FedADT: An Adaptive Method Based on Derivative Term for Federated Learning.

Area of Science:

• Machine Learning
• Distributed Systems
• Computer Vision

Background:

• Federated learning enables collaborative model training on local data.
• Challenges include slow convergence, lack of adaptivity, and noise sensitivity in standard federated learning.

Purpose of the Study:

• To introduce Federated Adaptive learning based on Derivative Term (FedADT) to address federated learning limitations.
• To enhance convergence speed and robustness against noise in distributed training.

Main Methods:

• FedADT integrates adaptive step sizes and gradient differences into local model updates.
• A moving average decay is applied to the derivative term to mitigate noise.
• Convergence analysis for non-convex objective functions is performed.

Main Results:

• FedADT achieves a convergence rate of 1/nT with appropriate hyperparameter tuning.
• Experiments on image classification (MNIST, Fashion MNIST) demonstrate FedADT's effectiveness.
• Receiver operating characteristic curves validate performance in clothing category prediction.

Conclusions:

• FedADT offers an effective solution for improving federated learning performance.
• The proposed method enhances convergence and noise robustness in distributed training environments.