An Adaptive Multiparameter Penalty Selection Method for Multiconstraint and Multiblock ADMM

Area of Science:

• Optimization Algorithms
• Numerical Analysis
• Applied Mathematics

Background:

• The alternating direction method of multipliers (ADMM) is a powerful algorithm for solving constrained optimization problems.
• ADMM's convergence rate is sensitive to the choice of its single penalty parameter, requiring careful tuning.
• Problems with multiple constraints or block matrix structures often exhibit slow convergence due to scale differences between constraints.

Purpose of the Study:

• To develop a new method for online selection of multiple penalty parameters for ADMM.
• To address the challenge of slow convergence in ADMM caused by scale differences among multiple constraints.
• To enhance the robustness and simplicity of ADMM implementations for complex optimization problems.

Main Methods:

• Proposes an online method to adaptively select multiple penalty parameters, one for each constraint.
• The method accounts for scale differences between constraints to improve ADMM convergence.
• Focuses on robustness to problem transformations and initial parameter choices.

Main Results:

• The proposed method adaptively manages scale differences between constraints.
• Numerical experiments show favorable performance compared to existing penalty parameter selection methods.
• The method demonstrates robustness and ease of implementation.

Conclusions:

• The novel online multi-parameter selection method improves ADMM convergence for problems with scaled constraints.
• This approach offers a practical and effective solution for enhancing ADMM performance in signal and image processing.
• The method provides a robust and simple alternative to traditional single-parameter tuning.