Generalized core vector machines.

Area of Science:

• Machine Learning
• Computational Statistics
• Optimization

Background:

• Kernel methods, including Support Vector Machines (SVM), are powerful machine learning techniques.
• Traditional SVM implementations face significant computational challenges (O(m3) time, O(m2) space) with large datasets due to their formulation as quadratic programming (QP) problems.
• Existing efficient approximations like the Core Vector Machine (CVM) have limitations, excluding popular methods such as Support Vector Regression (SVR).

Purpose of the Study:

• To develop a generalized Core Vector Machine (CVM) algorithm capable of handling a wider range of kernel methods and datasets.
• To overcome the scalability limitations of traditional Support Vector Machine (SVM) and Support Vector Regression (SVR) algorithms.
• To introduce a computationally efficient alternative for large-scale kernel-based machine learning.

Main Methods:

• Introduction of the center-constrained minimum enclosing ball (MEB) problem.
• Extension of the Core Vector Machine (CVM) algorithm based on the center-constrained MEB.
• Application of the generalized CVM to various kernel methods, including Support Vector Regression (SVR) and ranking SVM.

Main Results:

• The generalized CVM algorithm achieves linear asymptotic time complexity (O(m)) and space complexity independent of dataset size (m).
• The algorithm demonstrates applicability to any linear/nonlinear kernel and methods like SVR.
• Experimental results show comparable performance to state-of-the-art SVM and SVR, with faster training times and fewer support vectors on large datasets.

Conclusions:

• The generalized CVM provides a scalable and efficient solution for applying kernel methods to large datasets.
• This advancement significantly broadens the applicability of CVM beyond its original scope, including SVR.
• The algorithm offers a practical alternative for large-scale machine learning tasks, reducing computational burden and improving efficiency.