Distributed inexact Newton method with adaptive step sizes.

Area of Science:

• Distributed optimization
• Networked systems
• Machine learning

Background:

• Distributed optimization is crucial for large-scale problems.
• Existing methods face challenges with communication and computation costs.

Purpose of the Study:

• Introduce a novel method, DINAS (Distributed Inexact Newton method with Adaptive step size).
• Improve efficiency and reduce communication overhead in distributed optimization.

Main Methods:

• DINAS uses adaptive step sizes and reduced global parameter knowledge.
• It avoids local Hessian inverse calculations and Hessian communications.
• Convergence analysis for inexact Newton methods with adaptive step sizes is provided.

Main Results:

• DINAS achieves quadratic convergence (computation) and linear convergence (communication) for personalized optimization.
• The communication convergence rate is independent of network topology and local function conditions.
• DINAS converges to the global solution for consensus optimization problems.
• Numerical experiments show significant improvements over existing methods.

Conclusions:

• DINAS offers a more efficient and practical approach to distributed optimization.
• The method is robust across different network structures and problem complexities.
• Provides theoretical insights into adaptive step size methods.