A novel bounded loss framework for support vector machines.

Area of Science:

• Machine Learning
• Statistical Learning Theory

Background:

• Support Vector Machines (SVM) and Support Vector Regression (SVR) are powerful algorithms for classification and regression.
• Traditional SVM/SVR can be sensitive to outliers and data perturbations, limiting their robustness.
• Existing loss functions may not adequately address these robustness concerns.

Purpose of the Study:

• To introduce a novel bounded loss framework for SVM and SVR.
• To develop a bounded exponential quantile loss (L_eq-loss) enhancing robustness against outliers and resampling.
• To theoretically analyze the properties and derive generalization error bounds for the proposed models.

Main Methods:

• Devised a bounded exponential quantile loss (L_eq-loss) inspired by the Pinball loss.
• Constructed Enhanced Quantile Support Vector Machines (EQSVM) and Enhanced Quantile Support Vector Regression (EQSVR).
• Utilized the concave-convex procedure (CCCP) and ClipDCD algorithm for optimization.
• Derived influence functions, breakdown point lower bounds, and generalization error bounds using Rademacher complexity.

Main Results:

• L_eq-loss demonstrated enhanced robustness of SVM and SVR against outliers.
• EQSVM showed improved robustness to resampling compared to standard SVM.
• Influence functions were proven to be bounded, and breakdown point lower bounds reached 1/2.
• Generalization error bounds for EQSVM were derived.

Conclusions:

• The proposed L_eq-loss framework effectively enhances the robustness of SVM and SVR.
• EQSVM and EQSVR offer improved stability and reliability, particularly in the presence of noisy data.
• The theoretical analysis supports the practical effectiveness demonstrated through experiments.