Support vector machines with constraints for sparsity in the primal parameters.

Area of Science:

• Machine Learning
• Computational Statistics

Background:

• Support Vector Machines (SVMs) are powerful classification tools.
• Traditional SVMs may not inherently perform feature selection, leading to complex models.
• Sparse solutions in SVMs can improve interpretability and reduce computational cost.

Purpose of the Study:

• To introduce a new Support Vector Machine (SVM) formulation for achieving sparse solutions in primal parameters.
• To develop a novel method for feature selection directly integrated within the SVM framework.
• To enhance the efficiency and interpretability of SVM models by identifying and dropping irrelevant features.

Main Methods:

• A novel SVM formulation incorporating additional constraints to induce sparsity in primal parameters.
• Utilized a ν-SVM formulation where ν controls the fraction of features considered.
• Developed two versions: a 2-norm SVM and a computationally efficient 1-norm SVM.
• Extended the approach for multiclass classification and group feature selection.

Main Results:

• The proposed methods demonstrated effective feature selection capabilities on synthetic and real datasets.
• Performance was comparable or superior to existing state-of-the-art SVM-based feature selection techniques.
• The 1-norm SVM variant offered a reduced computational burden compared to the 2-norm version.

Conclusions:

• The novel SVM formulation provides an effective mechanism for sparse feature selection.
• The approach offers flexibility for multiclass problems and group feature selection.
• This method enhances SVMs for applications requiring interpretable and efficient feature selection.