A Self-Consistent-Field Iteration for Orthogonal Canonical Correlation Analysis.

Area of Science:

• Machine Learning
• Statistical Analysis
• Data Mining

Background:

• Orthogonal Canonical Correlation Analysis (OCCA) is crucial for visualization, pattern recognition, and feature extraction.
• Existing OCCA methods suffer from numerical instability or inefficiency.
• There is a need for robust and efficient OCCA algorithms.

Purpose of the Study:

• To propose an efficient and numerically stable algorithm for solving OCCA.
• To develop a new alternating numerical scheme based on trace-fractional maximization with orthogonality constraints.
• To extend the approach for orthogonal multiset CCA.

Main Methods:

• An alternating numerical scheme is proposed, centered on a sub-maximization problem within a trace-fractional form.
• A customized self-consistent-field (SCF) iteration is devised for the sub-maximization problem, proven to be globally convergent.
• A new trace-fractional maximization problem for orthogonal multiset CCA is formulated, utilizing Jacobi-style or Gauss-Seidel-style updates.

Main Results:

• The proposed SCF iteration guarantees global convergence to a KKT point.
• The alternating numerical scheme demonstrates consistent convergence.
• Experimental results show competitive or superior performance compared to existing methods in multi-label classification and multi-view feature extraction.
• The proposed algorithms are more efficient than existing approaches.

Conclusions:

• The developed algorithms provide an efficient and stable solution for OCCA and orthogonal multiset CCA.
• The SCF iteration and alternating scheme offer significant improvements over traditional methods.
• The approach is effective for real-world applications like multi-label classification and multi-view feature extraction.