Global Model Selection via Solution Paths for Robust Support Vector Machine.

Area of Science:

• Machine Learning
• Computational Statistics

Background:

• Robust Support Vector Machines (RSVM) with ramp loss outperform traditional Support Vector Machines (SVM) using hinge loss in generalization.
• Effective RSVM performance relies on optimal regularization and ramp parameters.
• Grid search for parameter selection is computationally intensive, particularly for fine-grained searches.

Purpose of the Study:

• To propose an efficient method for selecting optimal regularization and ramp parameters for RSVM.
• To address the high computational cost associated with traditional parameter tuning techniques.

Main Methods:

• Introducing Solution Paths of RSVM (SPRSVM) based on the concave-convex procedure (CCCP) to track non-convex RSVM solutions.
• Utilizing incremental and decremental learning algorithms to manage Karush-Khun-Tucker violating samples during solution path tracking.
• Leveraging piecewise linearity of the model function to compute error paths and identify optimal parameters.

Main Results:

• SPRSVM enables global searching for regularization and ramp parameters.
• Demonstrated significant reduction in computational time compared to grid search.
• Verified the finite convergence of SPRSVM and analyzed its computational complexity.

Conclusions:

• SPRSVM provides an efficient and effective approach for tuning RSVM parameters.
• The method significantly reduces computational burden, making RSVM more practical for real-world applications.
• SPRSVM ensures optimal parameter selection for improved model generalization.