Implementing Kernel Methods Incrementally by Incremental Nonlinear Projection Trick.

Area of Science:

• Machine Learning
• Kernel Methods
• Reproducing Kernel Hilbert Spaces

Background:

• The nonlinear projection trick (NPT) allows direct computation of sample coordinates in a reproducing kernel Hilbert space, extending machine learning algorithms to kernel versions without the traditional kernel trick.
• NPT faces challenges with incremental implementation due to the need to manage an ever-increasing kernel matrix with new training data.

Purpose of the Study:

• To propose an incremental version of the nonlinear projection trick (INPT).
• To enable efficient kernelized incremental learning by addressing the limitations of the original NPT.

Main Methods:

• The proposed INPT simplifies NPT by observing that the centerization step is unnecessary.
• This simplification allows INPT to avoid modifying the coordinates of existing data, facilitating integration with incremental algorithms.

Main Results:

• The INPT successfully implements incremental versions of kernel methods, including kernel singular value decomposition, kernel principal component analysis, and kernel discriminant analysis.
• Demonstrated effectiveness in kernel matrix reconstruction, letter classification, and face image retrieval tasks.

Conclusions:

• INPT provides an effective and efficient approach for incremental kernel learning.
• The method allows existing incremental algorithms to be directly kernelized without the computational burden of large kernel matrices.