Data reduction for SVM training using density-based border identification.

Area of Science:

• Machine Learning
• Data Science
• Pattern Recognition

Background:

• Support Vector Machines (SVMs) are widely used for classification and regression.
• SVM training is computationally expensive for large datasets due to quadratic programming optimization.
• Existing methods aim to reduce training data by selecting key instances, like support vectors.

Purpose of the Study:

• To present a novel density-based method, Density-based Border Identification (DBI), for reducing SVM training data.
• To explore variations of DBI and its application to higher-dimensional data using dimensionality reduction.
• To evaluate the effectiveness of DBI in minimizing training data size and improving computational efficiency.

Main Methods:

• Developed a density-based method (DBI) to extract border instances from datasets.
• Applied DBI to lower-dimensional embeddings (using Uniform Manifold Approximation and Projection - UMAP) for high-dimensional data.
• Evaluated multiple DBI variations on benchmark datasets like Banana, USPS, and Adult9a.

Main Results:

• DBI effectively reduced the size of training data by extracting border instances.
• The method achieved significant training and prediction speedups.
• Classification accuracy was maintained adequately compared to training on the full dataset.
• DBI showed competitive performance against state-of-the-art methods like BPLSH, CBCH, and SE.

Conclusions:

• The proposed Density-based Border Identification (DBI) method is effective in reducing SVM training data.
• DBI offers practical benefits in terms of computational efficiency and speedup for large datasets.
• The method shows potential for practical application in machine learning tasks involving large-scale SVM training.