Stochastic Primal-Dual Hybrid Gradient Algorithm with Adaptive Step Sizes

Area of Science:

• Optimization Algorithms
• Computational Science
• Applied Mathematics

Background:

• Stochastic primal-dual hybrid gradient (SPDHG) algorithms are widely used for large-scale convex optimization due to their scalability.
• Convergence of SPDHG relies on an upper bound for the product of primal and dual step sizes.
• Selecting optimal step size ratios for SPDHG remains a challenge in practical applications.

Purpose of the Study:

• To develop a novel primal-dual algorithm with adaptive step sizes for convex optimization.
• To introduce a general class of adaptive SPDHG (A-SPDHG) algorithms.
• To provide systematic strategies for selecting step sizes in SPDHG to ensure convergence.

Main Methods:

• Proposed a general class of adaptive SPDHG (A-SPDHG) algorithms.
• Proved convergence properties of A-SPDHG under weak assumptions.
• Developed concrete parameter-updating strategies for A-SPDHG.

Main Results:

• Demonstrated the convergence of the proposed A-SPDHG algorithms.
• Validated the effectiveness of the adaptive schemes through numerical examples.
• Showcased successful application in computed tomography reconstruction.

Conclusions:

• The proposed adaptive step size strategies for SPDHG algorithms enhance convergence and practical applicability.
• A-SPDHG offers a robust solution for large-scale convex optimization problems.
• The developed methods provide a systematic approach to step size selection, overcoming previous limitations.