Training sparse least squares support vector machines by the QR decomposition.

Area of Science:

• Machine Learning
• Computational Statistics

Background:

• Least Squares Support Vector Machines (LS-SVMs) often yield non-sparse solutions.
• This lack of sparsity can limit model interpretability and efficiency.

Purpose of the Study:

• To develop a sparse LS-SVM model by integrating the Kernel Matching Pursuit (KMP) algorithm.
• To address the non-sparseness problem inherent in standard LS-SVM formulations.

Main Methods:

• The Kernel Matching Pursuit (KMP) algorithm was revisited through the lens of QR decomposition of the kernel matrix.
• Support vectors were strategically selected to minimize leave-one-out cross-validation (LOOCV) error.
• Efficient and accurate computation of LOOCV for the sparse LS-SVM was demonstrated.

Main Results:

• The proposed sparse LS-SVM models achieved comparable accuracy to traditional SVM and other sparse LS-SVM variants.
• The KMP-based approach significantly enhanced model sparsity.
• Leave-one-out cross-validation was computed efficiently and accurately.

Conclusions:

• The KMP algorithm offers an effective regularization parameter for achieving sparsity in LS-SVMs.
• The developed sparse LS-SVM models provide a balance of high accuracy and improved sparsity.
• This approach enhances the practical applicability of LS-SVMs in various datasets.