Training Lp norm multiple kernel learning in the primal.

Area of Science:

• Machine Learning
• Optimization Algorithms
• Computational Statistics

Background:

• Multiple Kernel Learning (MKL) often uses alternating optimization, alternating between dual Support Vector Machines (SVMs) and kernel weight updates.
• Exploring primal optimization for Lp norm MKL is valuable for alternative computational approaches.
• The effectiveness of primal optimization for Laplacian SVMs (LapSVMs) is noted as superior to dual methods.

Purpose of the Study:

• To propose an Lp norm multiple kernel learning algorithm operating in the primal space.
• To leverage alternating optimization for solving SVMs in the primal and updating kernel weights.
• To investigate the suitability of primal MKL within manifold regularization frameworks.

Main Methods:

• An alternating optimization strategy is employed, involving primal SVM solutions via the preconditioned conjugate gradient method.
• Kernel weights are learned iteratively within the alternating optimization cycles.
• Theoretical analysis using empirical Rademacher complexity is conducted for primal MKL.

Main Results:

• The proposed primal MKL algorithm achieves analytical solutions for kernel weights.
• The method demonstrates enhanced effectiveness for manifold regularization tasks compared to dual approaches.
• Optimization of empirical Rademacher complexity yields specific kernel weight solutions.

Conclusions:

• The developed primal Lp norm MKL algorithm is feasible and effective, as shown by experimental results.
• The primal approach offers advantages for manifold regularization, particularly with LapSVMs.
• Theoretical insights into primal MKL connect empirical Rademacher complexity to kernel weight determination.