An Online Riemannian PCA for Stochastic Canonical Correlation Analysis

Area of Science:

• Statistics
• Machine Learning
• Numerical Optimization

Background:

• Canonical Correlation Analysis (CCA) is crucial for finding relationships between datasets.
• Existing CCA algorithms face limitations in computational complexity and the number of components extractable.
• Optimization on Riemannian manifolds offers advanced techniques for matrix problems.

Purpose of the Study:

• To introduce an efficient stochastic algorithm, RSG+, for Canonical Correlation Analysis (CCA).
• To improve the scalability and performance of CCA for extracting multiple components.
• To explore the application of the developed algorithm in training fair models.

Main Methods:

• Reparametrization of projection matrices into structured matrices.
• Leveraging numerical optimization techniques for Riemannian manifolds.
• Development of the RSG+ stochastic algorithm.

Main Results:

• The RSG+ algorithm achieves O(d^2 k) runtime complexity per iteration for extracting k components.
• Demonstrates a faster convergence rate compared to existing methods.
• Empirical experiments show promising performance on common datasets.

Conclusions:

• The RSG+ algorithm provides a significant improvement in efficiency for multi-component CCA.
• The reparametrization strategy effectively integrates Riemannian optimization.
• The method shows potential for applications in areas like fair machine learning with incomplete data.