Online kernel principal component analysis: a reduced-order model.

Area of Science:

• Machine Learning
• Data Science
• Dimensionality Reduction

Background:

• Kernel principal component analysis (kernel-PCA) is a powerful nonlinear extension of principal component analysis.
• Traditional kernel-PCA often requires a model order equal to the number of observations, limiting its online application.
• Efficient dimensionality reduction is crucial for analyzing large and complex datasets.

Purpose of the Study:

• To propose a novel online algorithm for kernel-PCA.
• To address the challenge of high model order in kernel-based methods for online processing.
• To develop a computationally efficient approach for nonlinear dimensionality reduction.

Main Methods:

• Developed a kernel-based version of Oja's rule for online kernel-PCA.
• Introduced a model order control mechanism for the online algorithm.
• Derived a recursive algorithm to extract multiple principal axes.
• Analyzed theoretical error bounds for the reduced-order model.

Main Results:

• The proposed online algorithm effectively performs kernel-PCA with controlled model order.
• Theoretical analysis provides an upper bound on approximation errors.
• Experimental validation on synthetic and handwritten digit datasets demonstrates effectiveness.
• The algorithm shows competitive or superior performance compared to classical and iterative kernel-PCA.

Conclusions:

• The developed online kernel-PCA algorithm offers an efficient and effective solution for nonlinear dimensionality reduction.
• Controlling model order is key to enabling online applications of kernel-PCA.
• The approach is suitable for real-world applications, including image analysis.