Algorithms for Sparse Support Vector Machines.

Area of Science:

• Machine Learning
• Computer Science
• Statistics

Background:

• Classification tasks often suffer from a high number of irrelevant features, hindering model performance and interpretability.
• Existing methods using L1 penalties for variable selection in support vector machines can bias parameter estimates and include superfluous features.

Purpose of the Study:

• To propose an alternative variable selection strategy using sparse-set constraints instead of traditional penalties.
• To develop and evaluate algorithms based on the proximal distance principle for achieving enhanced sparsity.

Main Methods:

• The study replaces conventional penalties with sparse-set constraints within the proximal distance principle.
• A novel objective function is formulated, incorporating the squared Euclidean distance to a sparsity set (k-non-zero components).
• Two algorithms are derived to implement this constrained optimization strategy.

Main Results:

• The proposed method effectively achieves greater feature sparsity compared to traditional penalty-based approaches.
• Simulated and real-world data examples demonstrate the efficacy of the new algorithms.
• The approach maintains classification power while significantly reducing feature dimensionality.

Conclusions:

• Sparse-set constraints offer a superior alternative for variable selection in classification, leading to improved sparsity.
• The developed algorithms provide a practical and effective means to implement this strategy.
• This research enhances model interpretability and efficiency by identifying crucial features accurately.