Fast ramp fraction loss SVM classifier with low computational complexity for pattern classification.

Area of Science:

• Machine Learning
• Pattern Classification
• Computational Complexity

Background:

• Support Vector Machines (SVMs) are effective for pattern classification but face computational challenges with large datasets.
• High computational complexity in traditional SVMs limits their application in extensive classification tasks.

Purpose of the Study:

• Introduce a novel Support Vector Machine (SVM) model, L_rf-SVM, to mitigate computational complexity.
• Achieve simultaneous sparsity and robustness in classification tasks.
• Develop a new optimality theory and an efficient algorithm for the L_rf-SVM model.

Main Methods:

• Developed a novel optimality theory for the nonsmooth and nonconvex L_rf-SVM using a proximal stationary point.
• Introduced an efficient alternating direction method of multipliers (ADMM) with a working set for L_rf-SVM.
• Ensured global convergence of the proposed algorithm.

Main Results:

• The L_rf-SVM algorithm demonstrates superior performance compared to nine other solvers.
• Achieved significant improvements in the number of support vectors, computation speed, and classification accuracy.
• Exhibited robustness to outliers and processed a dataset of over 10⁷ samples in 18.67 seconds.

Conclusions:

• The proposed L_rf-SVM model and its associated ADMM algorithm offer a computationally efficient solution for large-scale pattern classification.
• The method provides a notable enhancement in speed and accuracy, making it suitable for extensive datasets.
• L_rf-SVM effectively balances sparsity and robustness, outperforming existing state-of-the-art solvers.