Very sparse LSSVM reductions for large-scale data.

Area of Science:

• Machine Learning
• Computational Statistics

Background:

• Least Squares Support Vector Machines (LSSVM) offer performance comparable to Support Vector Machines (SVMs) but lack sparsity.
• This deficiency limits LSSVMs in handling large-scale datasets due to computational and memory constraints.

Purpose of the Study:

• To address the sparsity limitations of LSSVM and Primal Fixed-Size LSSVM (PFS-LSSVM).
• To develop a method for achieving highly sparse models without significant error increase, enabling scalability to large datasets.

Main Methods:

• Introduced a second level of sparsity using L0-norm-based reductions to iteratively sparsify LSSVM and PFS-LSSVM models.
• Utilized Nyström approximation with prototype vectors (PVs) in PFS-LSSVM to introduce initial sparsity.

Main Results:

• The proposed iterative sparsification method results in highly sparse LSSVM models.
• The approximations enabled scaling to large-scale datasets, overcoming memory and computational challenges.
• Experiments demonstrated that the developed approaches achieve sparse models with minimal impact on error rates.

Conclusions:

• The novel sparsification technique effectively reduces LSSVM and PFS-LSSVM models, making them suitable for large-scale applications.
• This method provides a significant advantage in computational efficiency and memory usage for machine learning tasks.