ANOTHER LOOK AT THE FAST ITERATIVE SHRINKAGE/THRESHOLDING ALGORITHM (FISTA).

Area of Science:

• Optimization Theory
• Applied Mathematics
• Machine Learning

Background:

• The Fast Iterative Shrinkage/Thresholding Algorithm (FISTA) is crucial for minimizing composite convex functions, particularly those with nonsmooth terms like L1 regularization.
• Existing FISTA methods are analyzed in relation to accelerated proximal gradient methods and worst-case cost function bounds.

Purpose of the Study:

• To develop a new, optimized approach for the Fast Iterative Shrinkage/Thresholding Algorithm (FISTA).
• To improve the minimization of composite convex functions with nonsmooth terms.

Main Methods:

• The study analyzes FISTA as an optimized acceleration of the proximal gradient method concerning worst-case cost function bounds.
• A novel algorithm is proposed by optimizing proximal gradient step coefficients based on worst-case composite gradient mapping bounds.
• Worst-case analysis using the Performance Estimation Problem framework underpins the theoretical proofs.

Main Results:

• Demonstrates that FISTA is an optimized acceleration strategy for proximal gradient methods regarding worst-case cost function bounds.
• Introduces a new algorithm derived from optimizing step coefficients for improved worst-case composite gradient mapping bounds.

Conclusions:

• The proposed method offers a new perspective on developing accelerated optimization algorithms.
• This work provides theoretical guarantees for the new algorithm's performance in minimizing composite convex functions.