Distributed Subgradient Method With Random Quantization and Flexible Weights: Convergence Analysis.

Area of Science:

• Optimization Algorithms
• Machine Learning Theory
• Distributed Systems

Background:

• Distributed subgradient (DSG) methods are crucial for large-scale optimization in machine learning.
• Existing DSG methods often assume perfect data communication, which raises privacy and feasibility concerns.
• Data quantization is a common solution but challenges algorithm convergence due to accuracy loss.

Purpose of the Study:

• To develop a robust DSG method that addresses convergence issues caused by data quantization.
• To analyze the convergence properties of the proposed method for various objective functions.
• To provide theoretical bounds on convergence rates influenced by quantization and network parameters.

Main Methods:

• Proposed a novel distributed subgradient method incorporating random quantization and flexible weights.
• Conducted theoretical analysis to derive convergence rate bounds for strongly and weakly convex functions.
• Performed numerical simulations in both convex and weakly convex settings to validate the findings.

Main Results:

• The proposed DSG method demonstrates improved convergence under random quantization.
• Derived upper bounds on convergence rates, considering quantization error, distortion, step sizes, and agent count.
• Analysis extends to weakly convex cases, offering broader applicability than prior work.

Conclusions:

• The novel DSG method effectively handles data quantization challenges in distributed optimization.
• Theoretical and numerical results confirm the algorithm's convergence and provide insights into its performance factors.
• This work advances the practical application of DSG methods in scenarios with imperfect communication.