Localized Multiple Kernel Learning Via Sample-Wise Alternating Optimization.

Area of Science:

• Machine Learning
• Computer Vision
• Optimization Algorithms

Background:

• Multiple Kernel Learning (MKL) methods face challenges in optimizing sample-specific kernel weights.
• Existing localized MKL (LMKL) approaches struggle with the non-convex nature of kernel weight updates.
• Standard MKL methods often have complexity tied to SVM solvers, limiting simultaneous optimization.

Purpose of the Study:

• To develop an effective and efficient alternating optimization method for training support vector machines (SVM)-based localized multiple kernel learning (LMKL).
• To address the challenge of optimizing sample-specific kernel weights in LMKL by decomposing the objective.
• To improve the generalization of LMKL by incorporating neighborhood information.

Main Methods:

• A novel primal-dual equivalence is used to decompose the canonical objective into sample-wise objectives.
• A sample-wise alternating optimization is performed, allowing independent optimization of localized kernel weights.
• Kernel weights are obtained via linear programming (l1-norm) or closed-form solutions (lp-norm).
• Neighborhood information is incorporated into the empirical loss for improved test-time generality.

Main Results:

• The proposed sample-wise alternating optimization method effectively trains LMKL models.
• The algorithm demonstrates efficiency in optimizing both kernel weights and the classifier simultaneously.
• Experiments show superior performance on benchmark machine learning and real-world computer vision datasets.

Conclusions:

• The developed sample-wise alternating optimization approach provides an effective solution for LMKL.
• The method enhances the performance and efficiency of SVM-based LMKL.
• Incorporating neighborhood information improves the generalization capabilities of the LMKL algorithm.