Support Vector Machine Classifier via L_0/1 Soft-Margin Loss.

Area of Science:

• Machine Learning
• Optimization Algorithms
• Computational Statistics

Background:

• Support Vector Machines (SVM) are widely used with various soft-margin losses.
• Existing methods often use surrogate losses for the ideal L0/1 soft-margin loss.
• Optimization algorithms for SVM are crucial for its practical applications.

Purpose of the Study:

• To address the challenge of optimizing the ideal L0/1 soft-margin loss SVM (L0/1-SVM).
• To establish a theoretical framework for L0/1-SVM, including optimality conditions.
• To develop an efficient algorithm for solving L0/1-SVM.

Main Methods:

• Developed optimality theory for L0/1-SVM, defining optimal solutions and P-stationary points.
• Introduced a rigorous definition of L0/1 support vectors and a working set.
• Proposed a fast alternating direction method of multipliers (ADMM) using the working set.

Main Results:

• Established the existence of optimal solutions and their relationship with P-stationary points for L0/1-SVM.
• The proposed ADMM converges to a locally optimal solution for L0/1-SVM.
• Numerical experiments showed the proposed method is faster and yields fewer support vectors than existing solvers.

Conclusions:

• The new theoretical framework and algorithm provide an effective way to solve L0/1-SVM.
• The method's efficiency increases with larger data sizes.
• This work advances SVM optimization, offering practical benefits in speed and model simplicity.