Discriminative Feature Selection via Employing Smooth and Robust Hinge Loss

Area of Science:

• Machine Learning
• Data Science
• Computer Science

Background:

• Existing sparsity-inducing feature selection methods often use regression-based loss functions, which are suboptimal for classification tasks.
• Multiclass hinge loss (HL) is effective for classification but challenging to optimize with sparsity regularization.

Purpose of the Study:

• To develop a new loss function that combines the benefits of regression and hinge loss for improved feature selection in classification.
• To introduce an optimized accelerated proximal gradient (APG) algorithm for efficient model training and enhanced class discrimination.

Main Methods:

• Proposed a novel smooth and robust hinge loss function for sparsity regularized feature selection models.
• Developed a new variant of the accelerated proximal gradient (APG) algorithm tailored for the proposed loss function.
• Introduced an efficient optimization technique for the proximal projection subproblem within the APG algorithm.

Main Results:

• The new APG algorithm demonstrates improved discriminative margins compared to standard APG methods.
• Theoretically proved optimal convergence rate of O(1/k^2) for the convex case of the new APG algorithm.
• Experimental results on nine datasets confirm the effectiveness of the proposed feature selection method.

Conclusions:

• The novel smooth and robust hinge loss effectively addresses limitations of existing methods for classification tasks.
• The proposed APG algorithm offers efficient optimization and superior performance in sparsity regularized feature selection.