On Selecting Effective Patterns for Fast Support Vector Regression Training.

Area of Science:

• Machine Learning
• Data Science

Background:

• Support Vector Regression (SVR) training is computationally intensive for large datasets, especially during parameter tuning.
• Reducing dataset size via pattern selection is a viable strategy to mitigate training time.

Purpose of the Study:

• To develop a fast pattern selection method for reducing the scale of large training datasets for SVR.
• To improve the efficiency of SVR model training without significant performance degradation.

Main Methods:

• A novel pattern selection approach is proposed, identifying k-nearest neighbors (kNNs) within local regions around target values.
• Patterns are retained based on the distribution of their nearest neighbors, prioritizing those outside the epsilon-tube.
• The method efficiently scans the entire training dataset by focusing on small local regions.

Main Results:

• The proposed method processed the Million Song Dataset (463,715 patterns) in under 10 seconds on standard hardware.
• It allows for predefined control over the size of the selected data subset.
• Empirical evaluations confirm significant elimination of redundant patterns, leading to faster SVR training with only a minor impact on performance.

Conclusions:

• The presented fast pattern selection method effectively reduces SVR training time for large-scale problems.
• This technique offers a practical solution for enhancing the scalability of SVR models.
• The method achieves efficiency while maintaining high predictive accuracy, making it suitable for real-world applications.