An Adaptive Approach to Learning Optimal Neighborhood Kernels

Area of Science:

• Machine Learning
• Kernel Methods
• Support Vector Machines

Background:

• Optimal kernel learning is crucial for kernel-based methods.
• Optimal Neighborhood Kernel Learning (ONKL) shows promise but lacks principled kernel specification.
• Existing methods struggle with defining the initial kernel for ONKL.

Purpose of the Study:

• To develop a principled approach for specifying the pre-specified kernel in ONKL.
• To jointly learn the pre-specified kernel, optimal neighborhood kernel, and SVM structure parameters.
• To enhance classification performance by adaptively identifying the optimal kernel.

Main Methods:

• Treating the pre-specified kernel as an extra variable for joint learning.
• Constraining the pre-specified kernel with a parameterized model to avoid trivial solutions.
• Employing the extended level method and Nesterov's method to solve the min-max optimization problem.

Main Results:

• The proposed approach adaptively identifies the pre-specified kernel.
• Demonstrated effectiveness with Gaussian radial basis function and multiple kernel learning instantiations.
• Achieved superior classification performance compared to original ONKL and related multiple kernel learning algorithms.

Conclusions:

• Jointly learning the pre-specified kernel enhances adaptive kernel identification.
• The method provides a probabilistic interpretation and unifies existing kernel learning approaches.
• The approach offers superior and adaptive classification performance on diverse datasets.