Robust Support Vector Machines for Classification with Nonconvex and Smooth Losses.

Area of Science:

• Machine Learning
• Statistical Learning Theory
• Robust Statistics

Background:

• Label noise presents a significant challenge in training accurate machine learning models.
• Large margin classifiers, like Support Vector Machines (SVMs), are sensitive to noisy labels.
• Existing methods often struggle to balance robustness and classification performance.

Purpose of the Study:

• To develop a robust large margin classifier resilient to label noise.
• To investigate the theoretical properties, including robustness and smoothness, of the proposed method.
• To provide an efficient algorithm for training the robust classifier.

Main Methods:

• Proposing robustified large margin support vector machines (RSVC) utilizing nonconvex classification losses.
• Interpreting robustness from a weighted viewpoint and demonstrating simultaneous smoothness.
• Drawing inspiration from M-estimation in robust statistics for loss function design.
• Developing an iteratively reweighted algorithm to solve the RSVC optimization problem, leveraging quadratic programming in the dual space.

Main Results:

• The proposed RSVC demonstrates enhanced robustness against label noise compared to standard methods.
• The classifier exhibits desirable smoothness properties due to the use of smooth classification losses.
• The iterative algorithm converges to a stationary point, ensuring efficient solution finding.
• Empirical validation on both artificial and real-world datasets confirms the effectiveness of RSVC.

Conclusions:

• Robustified large margin support vector machines (RSVC) offer a promising solution for learning classifiers in the presence of label noise.
• The method achieves a balance between robustness, smoothness, and generalization ability.
• The proposed iterative algorithm provides an efficient and practical approach for training RSVC models.