BOBA: Byzantine-Robust Federated Learning with Label Skewness.

Area of Science:

• Machine Learning
• Distributed Systems
• Cybersecurity

Background:

• Federated learning (FL) typically assumes independent and identically distributed (IID) data across clients.
• Existing robust aggregation rules (AGRs) are often designed for IID settings and struggle with non-IID data, specifically label skewness.
• Label skewness, where clients possess data from only a few classes, presents challenges like selection bias and increased vulnerability to Byzantine attacks in FL.

Purpose of the Study:

• To develop a robust aggregation method for federated learning under label-skewed, non-IID data distributions.
• To address the selection bias and enhanced vulnerability to Byzantine attacks inherent in current AGRs within non-IID settings.
• To propose an efficient and theoretically sound approach that improves model performance and fairness across all data classes.

Main Methods:

• We propose BOBA (Balanced and Optimized Byzantine-aware Aggregation), an efficient two-stage aggregation method.
• BOBA is designed to mitigate selection bias and enhance robustness against Byzantine adversaries in non-IID federated learning.
• Theoretical analysis confirms the convergence of BOBA with optimal error bounds.

Main Results:

• Empirical evaluations show BOBA significantly reduces performance drops for underrepresented classes caused by label skewness.
• BOBA demonstrates superior robustness against Byzantine attacks compared to state-of-the-art AGRs in non-IID settings.
• The method achieves unbiased aggregation and maintains high accuracy across diverse datasets and models.

Conclusions:

• BOBA effectively addresses the challenges of label skewness and Byzantine attacks in federated learning.
• The proposed method offers a practical and efficient solution for building more robust and fair federated models.
• Future work can explore extensions of BOBA to other forms of data heterogeneity.