Gradient-Refined Federated Learning on Head-Tail Imbalanced Data

Federated learning has emerged as a transformative paradigm for distributed data collaboration, facilitating knowledge aggregation across multiple local clients through a global server while rigorously preserving data privacy. However, its performance is significantly hindered by the global head-tail imbalance, where tail classes with scarce data are often dominated by head classes. This challenge, known as federated long-tailed learning, arises from the intrinsic conflict between class knowledge acquisition and privacy preservation. Existing methodologies falter in resolving this conflict, as the abstraction of data knowledge in federated communication complicates the extraction of class-level knowledge, resulting in imbalanced global models and diminished performance. To simultaneously address this imbalance and uphold privacy, we introduce FedGRE, a gradient-refined federated learning approach that constructs global gradients and facilitates refined global gradient descent. FedGRE enhances gradients through two pivotal mechanisms: accumulation diffusion and accumulation refinement. The former amalgamates accumulated gradients with stochastic gradient perturbations to alleviate class imbalance, while the latter utilizes the accumulation as an anchor to calibrate global gradient updates, ensuring consistency and mitigating oscillations. Additionally, we implement a consistency integration technique to incorporate the refined accumulation into the global model, guaranteeing privacy-preserving and class-balanced global optimization. Extensive experiments on six datasets demonstrate that FedGRE significantly outperforms 14 state-of-the-art (SOTA) methods in federated long-tailed classification while maintaining robust privacy protection.