Sparse LSSVM in Primal Using Cholesky Factorization for Large-Scale Problems.

Area of Science:

• Machine Learning
• Computational Statistics

Background:

• Least Squares Support Vector Machine (LSSVM) is a popular model due to its explicit solution.
• A key limitation of LSSVM is its lack of solution sparseness, hindering efficiency in large-scale problem training.

Purpose of the Study:

• To derive an equivalent LSSVM model in primal space (P-LSSVM).
• To develop algorithms for solving P-LSSVM, achieving sparse solutions for efficient large-scale training.

Main Methods:

• Derivation of P-LSSVM using the representer theorem.
• Development of two algorithms based on Cholesky factorization for sparse solutions.
• Kernel matrix decomposition and Hessian matrix approximation techniques.

Main Results:

• P-LSSVM can yield exact sparse solutions for low-rank kernel matrices.
• Algorithms achieve highly sparse solutions (<1% sparsity) with acceptable test accuracies on large-scale nonlinear problems.
• Theoretical analysis and experiments confirm P-LSSVM provides the sparsest solutions for linear learning.

Conclusions:

• The proposed P-LSSVM model and associated algorithms offer a more efficient approach for large-scale machine learning.
• The developed methods significantly improve computational efficiency by achieving high sparsity levels.
• This work presents a superior alternative for training large-scale Support Vector Machine problems.