Optimal projection method determination by Logdet Divergence and perturbed von-Neumann Divergence.

Area of Science:

• Machine Learning
• Kernel Methods
• Support Vector Machines

Background:

• Positive semi-definiteness is crucial for efficient Support Vector Machine (SVM) solutions via convex quadratic programming.
• Many real-world similarity functions generate indefinite kernels, posing challenges for standard SVM algorithms.

Purpose of the Study:

• To propose and evaluate a projection method for effectively utilizing indefinite kernels in SVMs.
• To investigate optimal parameter determination for the projection method within an unconstrained optimization framework.

Main Methods:

• Developed a projection method constructing a projection matrix for indefinite kernels.
• Generalized existing spectrum methods (denoising, flipping) into a unified projection framework.
• Applied Bregman matrix divergence theory for unconstrained optimization to determine the optimal parameter λ.
• Introduced a perturbed von-Neumann divergence for measuring kernel relationships and optimizing λ.

Main Results:

• The projection method demonstrated comparable or superior performance to existing indefinite kernel methods on diverse datasets.
• Comparing optimal λ determination methods, Logdet divergence yielded near-optimal performance for the projection method.
• Perturbed von-Neumann divergence also proved effective in determining a suitable optimal projection method parameter.

Conclusions:

• The projection method provides an accessible approach for handling indefinite kernels in SVMs.
• Parameter optimization via unconstrained optimization under Bregman matrix divergence offers a novel strategy for kernel SVMs.
• This work may open new avenues for kernel SVM applications with diverse objectives.