Online Stochastic DCA With Applications to Principal Component Analysis.

Area of Science:

• Optimization
• Machine Learning
• Data Science

Background:

• Stochastic algorithms are vital for big data processing.
• Nonsmooth stochastic Difference-of-Convex (DC) programs are a key class of nonconvex optimization problems with broad applications, especially in machine learning.
• Existing methods may not be optimal for continuously generated, streaming data.

Purpose of the Study:

• To develop novel online stochastic algorithms for nonsmooth DC programs.
• To adapt the powerful DC Algorithm (DCA) for the online, streaming data context.
• To provide rigorous convergence analysis for the proposed algorithms.

Main Methods:

• The study proposes new online stochastic algorithms.
• These algorithms are based on the Difference-of-Convex (DC) Algorithm (DCA) framework.
• Stochastic approximations (SAs) are employed, replacing deterministic DCA quantities with noisy estimators from new data samples.

Main Results:

• New online stochastic algorithms for nonsmooth DC programs are presented.
• Convergence analysis is rigorously established using convex analysis and martingale theory.
• The algorithms' efficacy is demonstrated on the expected principal component analysis (PCA) problem in machine learning.

Conclusions:

• The developed algorithms offer an effective approach for nonconvex stochastic optimization with streaming data.
• The theoretical analysis provides a strong foundation for the algorithms' performance.
• The application to PCA highlights practical utility in machine learning.