Riemannian Acceleration for Sparse PCA With Separable Structure and Second-Order Information Exploration

Area of Science:

• Computational statistics
• Machine learning
• Data science

Background:

• Sparse Principal Component Analysis (SPCA) is crucial for high-dimensional data analysis.
• Existing SPCA methods face computational challenges.
• Efficient algorithms are needed for practical applications.

Purpose of the Study:

• To develop an efficient and robust algorithm for SPCA.
• To address the computational intensity of SPCA.
• To uncover novel structures within the SPCA problem.

Main Methods:

• Introduced SPCA ACC, an algorithm leveraging Variable Projection (VP).
• Generalized VP to separable nonlinear problems on the Stiefel manifold.
• Resolved parameter coupling using a second-order Riemannian accelerated VP strategy.

Main Results:

• SPCA ACC optimizes in a lower-dimensional parameter space.
• The algorithm demonstrates rapid convergence.
• Significant reductions in computational costs were observed.

Conclusions:

• SPCA ACC offers a theoretically sound and practically efficient solution for SPCA.
• The method achieves local quadratic convergence.
• Validated through numerical experiments on diverse datasets.