Applications of Spectral Gradient Algorithm for Solving Matrix ℓ2,1-Norm Minimization Problems in Machine Learning.

Area of Science:

• Optimization
• Machine Learning
• Numerical Analysis

Background:

• Matrix norm regularization is crucial for multi-task learning, enabling the capture of joint features across tasks.
• Convex minimization problems are fundamental in various scientific and engineering domains.

Purpose of the Study:

• To propose, analyze, and test a new spectral gradient algorithm for solving convex minimization problems.
• Specifically targeting matrix $\ell_{2,1}$-norm regularized least squares problems common in multi-task learning.

Main Methods:

• Minimizing a quadratic approximated model to derive a search direction.
• Incorporating a nonmonotone line search to enhance numerical performance.
• Analyzing convergence properties under mild conditions.

Main Results:

• The derived search direction ensures automatic descent and simplifies to the spectral gradient direction without regularization.
• The proposed algorithm is shown to converge to a critical point.
• Experimental results on synthetic data validate the algorithm's effectiveness and superiority over existing methods.

Conclusions:

• The spectral gradient algorithm offers an easily performable method requiring only gradient and function values.
• It effectively addresses matrix $\ell_{2,1}$-norm regularized least squares problems in multi-task learning.
• The algorithm demonstrates competitive performance and potential for broader applications in convex optimization.