Heterogeneity-aware high-efficiency federated learning with hybrid synchronous-asynchronous splitting strategy

Area of Science:

• Artificial Intelligence
• Machine Learning
• Distributed Systems

Background:

• Federated Learning (FL) enables collaborative model training while preserving data privacy.
• System heterogeneity in FL leads to the 'straggler issue,' delaying aggregation due to resource-limited devices.
• Current solutions often neglect edge constraints, risking model bias and data omission.

Purpose of the Study:

• To introduce HA-HEFL, a Heterogeneity-Aware High-Efficiency Federated Learning framework.
• To address the trade-offs between training efficiency, model accuracy, and resource consumption in FL.
• To mitigate the negative impacts of system heterogeneity on FL performance.

Main Methods:

• Resource-aware adaptive model customization using neuron-level profiling and priority-based selection.
• Hybrid synchronous-asynchronous split training with knowledge distillation for feature extraction and classifier updates.
• Baseline-prioritized weighted aggregation to ensure balanced global model updates.

Main Results:

• HA-HEFL significantly improves convergence speed compared to existing methods.
• The framework enhances overall model accuracy.
• HA-HEFL demonstrates a notable reduction in network traffic.

Conclusions:

• HA-HEFL effectively balances training efficiency, model accuracy, and resource consumption in heterogeneous FL environments.
• The proposed adaptive model customization and hybrid training strategy overcome straggler issues.
• HA-HEFL offers a superior approach for practical FL deployments with diverse edge devices.