A new randomized Kaczmarz based kernel canonical correlation analysis algorithm with applications to information...

Area of Science:

• Statistics
• Machine Learning
• Data Science

Background:

• Canonical Correlation Analysis (CCA) identifies linear relationships between multivariate variable sets.
• Kernel CCA extends CCA to capture nonlinear relationships but can suffer from overfitting.
• High-dimensional data feature selection remains a challenge for existing kernel CCA methods.

Purpose of the Study:

• To develop a novel kernel CCA algorithm addressing overfitting in high-dimensional data.
• To analyze the theoretical convergence properties of the proposed algorithm.
• To establish a lower bound for the minimum number of iterations required.

Main Methods:

• A new kernel CCA algorithm is proposed utilizing the randomized Kaczmarz method.
• Theoretical convergence is analyzed using the scaled condition number.
• The algorithm's efficiency is evaluated through numerical experiments.

Main Results:

• The developed kernel CCA algorithm demonstrates effectiveness on synthetic and real-world datasets.
• Performance is validated in cross-language document retrieval and content-based image retrieval tasks.
• Numerical results show the algorithm is competitive with state-of-the-art kernel CCA methods.

Conclusions:

• The randomized Kaczmarz-based kernel CCA offers an effective solution for nonlinear dimensionality reduction and feature selection.
• The algorithm exhibits strong performance and efficiency, outperforming or matching existing methods.
• This work contributes a theoretically grounded and practically effective kernel CCA approach.