Performance Enhancement in Federated Learning by Reducing Class Imbalance of Non-IID Data.

Area of Science:

• Machine Learning
• Distributed Systems
• Data Science

Background:

• Federated learning (FL) clients often use non-independent and identically distributed (non-IID) datasets, leading to significant performance degradation.
• Addressing data heterogeneity is crucial for effective federated learning model training.

Purpose of the Study:

• To propose an efficient algorithm that enhances federated learning performance by mitigating the negative impacts of non-IID datasets.
• To improve model accuracy and reduce computational and communication overhead in federated learning.

Main Methods:

• Reducing intra-client class imbalance by aligning client class distributions towards a uniform distribution.
• Selecting clients for participation to ensure their combined class distribution approximates a uniform distribution, thus mitigating inter-client imbalance.
• Dynamically adjusting local training data amounts, batch sizes, and learning rates based on effective local dataset sizes.

Main Results:

• The proposed algorithm achieved a 20% higher accuracy compared to existing federated learning algorithms on CIFAR-10 and MNIST datasets.
• Demonstrated significant accuracy improvements while utilizing fewer computational and communication resources.
• Showcased reduced data usage and fewer participating clients during training.

Conclusions:

• The developed algorithm effectively overcomes the challenges posed by non-IID data in federated learning.
• Offers a more efficient and accurate approach to federated learning, suitable for real-world distributed data scenarios.
• Provides a practical solution for enhancing federated learning performance with reduced resource demands.