Soft margin multiple kernel learning.

Area of Science:

• Machine Learning
• Artificial Intelligence
• Computer Science

Background:

• Traditional L1 Multiple Kernel Learning (MKL) methods sometimes underperform compared to simpler averaging techniques.
• Existing MKL approaches lack flexibility in handling different loss functions and constraints.
• There is a need for improved MKL frameworks that are both effective and computationally efficient.

Purpose of the Study:

• To propose a novel soft margin framework for Multiple Kernel Learning (MKL).
• To enhance the effectiveness and sparsity of kernel combination methods.
• To unify and generalize existing MKL approaches within a single framework.

Main Methods:

• Introduced a kernel slack variable into the quadratic constraints of MKL.
• Developed a soft margin framework accommodating hinge loss, square hinge loss, and square loss functions.
• Proposed efficient algorithms for solving the hinge loss and square hinge loss soft margin MKL objectives.

Main Results:

• The proposed soft margin framework unifies various existing MKL methods, including average kernel, L1MKL, and L2MKL.
• Demonstrated that different hyper-parameter settings bridge the gap between average kernel, L1MKL, and the proposed hinge loss soft margin MKL.
• Experimental results on benchmark datasets and real-world applications (video/event recognition) show effective and sparse solutions.

Conclusions:

• The novel soft margin MKL framework provides a unified and more effective approach to kernel combination.
• The proposed methods achieve efficient and sparse solutions, outperforming traditional MKL techniques in practical scenarios.
• This framework offers a flexible foundation for developing advanced MKL algorithms.