Gradient-Refined Federated Learning on Head-Tail Imbalanced Data | JoVE Visualize

Area of Science:

Artificial Intelligence
Machine Learning
Distributed Systems

Background:

Federated learning enables collaborative model training across decentralized clients while preserving data privacy.
A key challenge is federated long-tailed learning, caused by imbalanced data distributions (head-tail imbalance) that hinder model performance.
Existing methods struggle to balance class knowledge acquisition with privacy preservation, leading to suboptimal global models.

Purpose of the Study:

To introduce FedGRE (Federated Gradient Refinement), a novel approach to address the challenges of federated long-tailed learning.
To develop a method that simultaneously tackles data imbalance and maintains rigorous data privacy in federated learning.
To enhance the extraction of class-level knowledge in federated settings without compromising privacy.

Main Methods:

FedGRE constructs global gradients using two mechanisms: accumulation diffusion and accumulation refinement.
Accumulation diffusion merges accumulated gradients with stochastic perturbations to mitigate class imbalance.
Accumulation refinement uses accumulated gradients as anchors for calibrated global updates, ensuring consistency and reducing oscillations. A consistency integration technique incorporates refined gradients into the global model.

Main Results:

FedGRE significantly outperforms 14 state-of-the-art methods on six benchmark datasets for federated long-tailed classification.
The proposed method demonstrates robust privacy protection alongside improved classification accuracy.
Experiments confirm FedGRE's effectiveness in achieving class-balanced global optimization.

Conclusions:

FedGRE effectively resolves the conflict between class knowledge acquisition and privacy preservation in federated learning.
The gradient refinement strategy in FedGRE leads to superior performance in federated long-tailed learning scenarios.
FedGRE offers a promising solution for building accurate and privacy-preserving federated models on imbalanced datasets.