The Proxy Step-Size Technique for Regularized Optimization on the Sphere Manifold.

Area of Science:

• Optimization Theory
• Computer Vision
• Applied Mathematics

Background:

• Optimization problems with constraints are common in machine learning and computer vision.
• Regularization techniques like norm regularization are crucial for model performance.
• Existing methods may face challenges with non-smooth convex functions on manifolds.

Purpose of the Study:

• To develop an effective and efficient solution for regularized optimization problems on the unit sphere.
• To introduce and analyze a novel 'proxy step-size' concept for Riemannian proximal gradient methods.
• To demonstrate the practical utility of the proposed method in computer vision applications.

Main Methods:

• The study proposes a Riemannian proximal gradient method incorporating a unique proxy step-size.
• The proxy step-size is proven to be monotone and determines the actual step-size and tangent update in closed-form.
• A line-search technique based solely on the smooth cost function guides the convergence.

Main Results:

• The proposed method is proven to converge to a critical point.
• The method demonstrates effectiveness when applied to nuclear norm, l1 norm, and nuclear-spectral norm regularization.
• Numerical experiments on three classical computer vision problems show consistent improvements.

Conclusions:

• The novel Riemannian proximal gradient method with proxy step-size offers an effective solution for constrained optimization.
• The method is computationally efficient and easy to implement.
• The approach shows significant promise for regularization in computer vision and related fields.