An effective method of pruning support vector machine classifiers.

Area of Science:

• Machine Learning
• Computational Science

Background:

• Support Vector Machine (SVM) classifiers can suffer from high computational costs and overfitting due to a large number of support vectors (SVs).
• Existing methods may struggle with discerning SVs in high-dimensional spaces or involve local minima.

Purpose of the Study:

• To develop a practical and effective method for pruning SVM classifiers.
• To reduce computational complexity and mitigate overfitting in trained SVM models.

Main Methods:

• Organizing kernel row vectors (corresponding to SVs) into clusters.
• Employing orthogonal projections (OPs) in the first phase to identify approximable kernel row vectors.
• Implementing crosswise propagations using OP coefficients in the second phase to remove redundant vectors within clusters.

Main Results:

• Pruning 42% of SVs resulted in an average classification accuracy change of only -0.7%.
• The average computation time for removing one SV was reduced to 0.006 of the training time.
• In some cases, over 90% of SVs were pruned with less than 0.1% reduction in accuracy.

Conclusions:

• Large numbers of superabundant SVs exist in trained SVMs, indicating significant potential for model optimization.
• The proposed pruning method is effective, efficient, and practical for upgrading existing SVM applications.
• A synergistic approach combining training and pruning is suggested for practical SVM deployment.