Towards Efficient Federated Learning: Layer-Wise Pruning-Quantization Scheme and Coding Design.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Distributed Systems

Background:

• Federated learning (FL) is a distributed machine learning paradigm enabling decentralized training.
• Practical FL deployments encounter significant communication-computation bottlenecks, hindering efficiency.
• Existing FL optimization methods often lack a unified approach to pruning and quantization.

Purpose of the Study:

• To propose FedLP-Q, a novel federated learning with layer-wise pruning-quantization scheme.
• To address communication-computation bottlenecks in FL systems.
• To develop a universal and explicit framework for FL optimization.

Main Methods:

• Developed FedLP-Q, a layer-wise pruning-quantization framework for FL.
• Designed specific pruning strategies for homogeneous and heterogeneous scenarios.
• Implemented a stochastic quantization rule and a corresponding coding scheme.

Main Results:

• FedLP-Q demonstrated improved system efficiency in communication and computation.
• The proposed scheme achieved controllable performance degradation.
• Theoretical and experimental evaluations validated the effectiveness of FedLP-Q.

Conclusions:

• FedLP-Q offers a joint pruning-quantization solution for FL systems.
• The layer-wise processing enables easy application in practical FL deployments.
• This framework effectively mitigates communication-computation bottlenecks in federated learning.