A Study of Enhancing Federated Learning on Non-IID Data with Server Learning

Area of Science:

• Machine Learning
• Distributed Systems
• Artificial Intelligence

Background:

• Federated Learning (FL) enables distributed model training using decentralized data.
• FL performance degrades significantly with non-independent and identically distributed (non-IID) client data, leading to poor accuracy and slow convergence.
• Existing methods struggle to fully address the challenges posed by non-IID data in FL.

Purpose of the Study:

• To investigate auxiliary server learning as a complementary strategy to enhance FL performance on non-IID data.
• To analyze the effectiveness of auxiliary server learning in improving model accuracy and convergence speed.
• To evaluate the approach's robustness with varying server dataset sizes and distributions.

Main Methods:

• Proposed auxiliary server learning as a method to augment FL training.
• Conducted theoretical analysis to understand the approach's impact on FL dynamics.
• Performed empirical experiments to validate performance improvements on non-IID datasets.

Main Results:

• Auxiliary server learning significantly improves model accuracy in FL settings with non-IID data.
• The approach accelerates convergence time, reducing the overall training duration.
• Performance gains are observed even with small server datasets that differ in distribution from client data.
• Auxiliary server learning complements existing techniques for mitigating non-IID data challenges in FL.

Conclusions:

• Auxiliary server learning is an effective complementary strategy for improving federated learning on non-IID data.
• The method offers substantial benefits in both accuracy and convergence speed.
• This approach shows promise for enhancing the practical applicability of FL in real-world scenarios with heterogeneous data.