Federated Collaborative Learning with Sparse Gradients for Heterogeneous Data on Resource-Constrained Devices.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Distributed Systems

Background:

• Federated learning (FL) enables collaborative model training on decentralized devices while preserving data privacy.
• Resource constraints on Internet of Things (IoT) devices limit the feasibility of training large-scale models.
• Federated split learning offers parallel training but suffers from client dependency and high communication overhead, especially with heterogeneous data.

Purpose of the Study:

• To design a sparse gradient collaborative federated learning model tailored for heterogeneous data on resource-constrained devices.
• To improve training efficiency and model applicability across diverse client distributions.
• To address the limitations of existing federated split learning frameworks.

Main Methods:

• Introduced a sparse gradient strategy using a position mask to minimize communication traffic.
• Implemented a dequantization strategy to restore gradient tensor accuracy.
• Developed an adaptive weighting strategy based on Euclidean distance to measure client influence on the global model.
• Combined sparse gradient quantization with adaptive weighting for a collaborative federated learning algorithm.

Main Results:

• The proposed algorithm significantly reduces communication traffic through sparse gradient selection.
• Adaptive weighting effectively accounts for client data heterogeneity, improving global model performance.
• The method achieves high classification efficiency on resource-constrained devices with heterogeneous data distributions.
• Demonstrated superior performance compared to existing federated learning approaches in challenging scenarios.

Conclusions:

• The developed sparse gradient collaborative federated learning model effectively addresses the challenges of training on heterogeneous, resource-constrained devices.
• The combination of sparse gradients and adaptive weighting enhances both efficiency and accuracy in federated learning.
• This approach offers a practical solution for deploying advanced machine learning models on edge devices.