Stochastic DCA for minimizing a large sum of DC functions with application to multi-class logistic regression

Area of Science:

• Optimization Theory
• Machine Learning
• Stochastic Processes

Background:

• Minimizing sums of Difference of Convex (DC) functions is a significant challenge in various fields, including stochastic optimization and machine learning.
• Existing methods may lack efficiency or guaranteed convergence for large-scale DC programming problems.

Purpose of the Study:

• To propose and analyze novel algorithms for large-scale DC function minimization.
• To address the specific application of group variable selection in multi-class logistic regression within multi-task learning.

Main Methods:

• Development of two DC Algorithm (DCA) based methods: stochastic DCA and inexact stochastic DCA.
• Theoretical analysis proving convergence to a critical point with probability one for both algorithms.
• Application and adaptation of stochastic DCA for group variable selection in multi-class logistic regression.

Main Results:

• Convergence of both stochastic DCA and inexact stochastic DCA to a critical point is guaranteed with probability one.
• The developed stochastic DCA for multi-class logistic regression is computationally inexpensive with explicit calculations.
• Numerical experiments demonstrate superior performance over existing methods in terms of classification accuracy, solution sparsity, and running time.

Conclusions:

• The proposed stochastic DCA algorithms provide efficient and reliable solutions for large-scale DC function minimization.
• These algorithms show significant promise for applications in multi-task learning, particularly in variable selection for logistic regression.
• The findings highlight the practical efficiency and effectiveness of the developed methods on benchmark and synthetic datasets.