Density-Dependent Quantized Least Squares Support Vector Machine for Large Data Sets.

Area of Science:

• Machine Learning
• Data Mining
• Computational Statistics

Background:

• Input data distribution is crucial for effective machine learning.
• Sparse data presents challenges in computational efficiency and scalability.
• Existing methods may not optimally handle data density variations.

Purpose of the Study:

• To propose a novel data density-dependent quantization scheme (DQS) for sparse input data representation.
• To integrate DQS with least squares support vector machine (LS-SVM) for enhanced performance on large datasets.
• To achieve significant sample size reduction and computational cost savings.

Main Methods:

• Development of a data density-dependent quantization scheme (DQS) using a single shrinkage threshold.
• Application of DQS as a preprocessing step for sparse data.
• Integration of quantized data with LS-SVM, utilizing the Nyström method for feature approximation.
• Development of a data density-dependent quantized LS-SVM (DQLS-SVM) with an analytic solution.

Main Results:

• DQS effectively quantizes large datasets into smaller subsets, achieving considerable sample size reduction.
• The Nyström method on quantized subsets significantly saves computational cost.
• The developed DQLS-SVM demonstrates high computational efficiency and good generalization performance on synthetic and benchmark datasets.
• DQLS-SVM provides an analytic solution in the primal solution space.

Conclusions:

• The proposed DQS is an effective method for sparse data representation and preprocessing.
• Integrating DQS with LS-SVM (DQLS-SVM) enhances computational efficiency and generalization for big data applications.
• DQS offers a viable approach for managing large, sparse datasets in machine learning.