Accelerated Mini-batch Randomized Block Coordinate Descent Method.

Area of Science:

• Optimization
• Machine Learning
• Computational Science

Background:

• Empirical risk minimization is crucial for machine learning model training.
• Regularized optimization problems often involve smooth and nonsmooth components.
• Block separable regularization allows for block coordinate descent methods.

Purpose of the Study:

• To develop a computationally efficient optimization method for regularized empirical risk minimization.
• To address the limitations of existing batch randomized block coordinate descent (RBCD) methods.
• To accelerate optimization using mini-batch data and semi-stochastic schemes.

Main Methods:

• Proposing a mini-batch randomized block coordinate descent (MRBCD) algorithm.
• Estimating partial gradients using randomly sampled mini-batches.
• Incorporating a semi-stochastic optimization scheme to reduce gradient variance.

Main Results:

• Theoretically demonstrating lower iteration complexity for MRBCD on strongly convex functions.
• Applying MRBCD to regularized sparse learning problems.
• Achieving superior computational performance compared to existing methods in numerical experiments.

Conclusions:

• MRBCD offers a more computationally feasible approach to large-scale optimization problems.
• The method effectively leverages sparsity for improved performance in machine learning.
• MRBCD presents a promising alternative for training regularized models efficiently.