Order-Optimal Byzantine-Robust Learning Under Heterogeneity via Fair Gradient Clipping.

Area of Science:

• Machine Learning
• Distributed Systems
• Cybersecurity

Background:

• Federated learning (FL) is vulnerable to Byzantine attacks that disrupt machine learning (ML) algorithm convergence.
• Existing robust aggregation rules often struggle with heterogeneous data or lack theoretical guarantees.
• Current methods may increase computational load or lack breakdown point analysis.

Purpose of the Study:

• To propose a new, theoretically sound aggregation rule for Byzantine-robust FL.
• To address the performance degradation of robust rules under data heterogeneity.
• To achieve optimal Byzantine-robust training error with reduced computational complexity.

Main Methods:

• A novel aggregation rule is introduced that clips worker gradients based on their distance from the aggregation center.
• Theoretical analysis is performed to determine the breakdown point of the proposed rule.
• Experimental validation is conducted to compare the new rule against existing median-based schemes.

Main Results:

• The proposed rule achieves a breakdown point of 0.5, the maximum for robust aggregators.
• The method demonstrates order-optimal Byzantine-robust training error, outperforming coordinate-wise median (CM) and geometric median (GM) under data heterogeneity.
• Experimental results confirm the mechanism's effectiveness against various attacks and its superiority over existing rules.

Conclusions:

• The devised gradient clipping aggregation rule offers superior Byzantine robustness and training accuracy in federated learning, especially with heterogeneous data.
• This approach provides a theoretically guaranteed, computationally efficient, and practically effective solution for secure distributed machine learning.
• The findings advance the field of robust federated learning by offering a more resilient and performant aggregation strategy.