SVRG-MKL: A Fast and Scalable Multiple Kernel Learning Solution for Features Combination in Multi-Class...

Area of Science:

• Computer Science
• Machine Learning
• Pattern Recognition

Background:

• Multiple Kernel Learning (MKL) models optimally combine features but struggle with large datasets due to computational and memory constraints.
• Existing MKL methods are limited to tens of thousands of samples, hindering their application in real-world scenarios.

Purpose of the Study:

• To propose a novel, scalable MKL strategy (SVRG-MKL) for efficiently combining compact descriptors in recognition tasks.
• To address the scalability limitations of traditional MKL models, enabling processing of millions of samples.

Main Methods:

• Formulated the problem using Multiple Kernel Learning (MKL) and solved it with a Stochastic Variance Reduced Gradient (SVRG) approach.
• Developed SVRG-MKL, which operates directly in the primal space to avoid Gram matrix computations and uses a linear complexity optimization algorithm.
• Introduced a modified SVRG algorithm where each kernel is treated distinctly for faster convergence.

Main Results:

• SVRG-MKL demonstrates inherent scalability, effectively combining descriptors from millions of samples.
• The proposed method achieves higher accuracy and significant speedup compared to existing MKL techniques on benchmark datasets.
• Experimental validation confirms the efficiency and effectiveness of SVRG-MKL.

Conclusions:

• SVRG-MKL offers a computationally efficient and scalable solution for combining multiple descriptors in recognition tasks.
• The technique can be extended to various MKL problems, including visual search, transfer learning, and group-sensitive/localized MKL formulations.