Efficient sparse generalized multiple kernel learning.

Area of Science:

• Machine Learning
• Kernel Methods
• Optimization

Background:

• Kernel methods are vital for data similarity representation in various applications.
• Multiple Kernel Learning (MKL) aims to find optimal kernel combinations.
• Existing MKL methods with L(1)-norm constraints yield sparse solutions but may discard useful kernels, while L(p)-norm (p>1) constraints retain information but risk noise sensitivity.

Purpose of the Study:

• To propose a Generalized Multiple Kernel Learning (GMKL) model that overcomes limitations of existing MKL approaches.
• To develop a novel constraint combining L(1)-norm and L(2)-norm for kernel weights.
• To enhance kernel representation by balancing sparsity and information preservation.

Main Methods:

• Introduced an elastic-net-type constraint on kernel weights, a linear combination of L(1)-norm and squared L(2)-norm.
• Formulated GMKL as a convex optimization problem solvable globally.
• Developed a level method for efficient optimization of the GMKL model.

Main Results:

• The proposed GMKL model effectively balances sparsity and retains information from base kernels.
• GMKL demonstrates a grouping effect, beneficial for feature selection.
• Experimental results on synthetic and real-world datasets confirm the model's effectiveness and efficiency.

Conclusions:

• GMKL offers a superior approach to Multiple Kernel Learning by addressing the trade-offs of existing L(1) and L(p) norm constraints.
• The model provides a flexible framework where L(1) and L(2) norm constrained MKL are special cases.
• GMKL enhances generalization performance and offers efficient optimization.