Learning SVM in Kreĭn Spaces.

Area of Science:

• Machine Learning
• Kernel Methods
• Linear Algebra

Background:

• Existing Support Vector Machine (SVM) solvers for Kreĭn spaces rely on matrix correction, non-convex minimization, or feature-space embedding.
• These methods face challenges with indefinite matrices, particularly those with numerous or large negative eigenvalues.

Purpose of the Study:

• To develop a theoretical foundation and practical solution for SVM solvers operating directly within Kreĭn spaces using indefinite similarity measures.
• To establish a direct correspondence between stabilization and minimization problems for SVMs in Kreĭn spaces.

Main Methods:

• A stabilization procedure is employed to derive a solution within the original Kreĭn space.
• A mathematical link is established between the stabilization problem and a classical, easily solvable minimization problem.
• Simple equations are provided for bidirectional conversion between these problem formulations.

Main Results:

• The proposed method, Kernel SVM (KSVM), successfully solves SVM problems with troublesome kernels (e.g., large numbers of negative eigenvalues).
• Experimental results demonstrate that KSVM outperforms previous approaches for handling indefinite matrices in SVM-like kernel methods.
• The theoretical foundation justifies the use of the original indefinite similarity measure in the Kreĭn space.

Conclusions:

• KSVM offers a robust and efficient solution for SVMs in Kreĭn spaces, particularly for kernels with indefinite properties.
• The direct approach in the original Kreĭn space avoids complex transformations and provides superior performance.
• This work bridges the gap between stabilization and minimization paradigms in kernel methods.