Nonparallel support vector regression model and its SMO-type solver.

Area of Science:

• Machine Learning
• Computational Statistics

Background:

• Twin Support Vector Regression (TSVR) is widely studied, but often overlooks structural risk minimization (SRM) and model sparseness.
• Existing TSVR variants have limitations in addressing these core principles effectively.

Purpose of the Study:

• Propose a novel Nonparallel Support Vector Regression (NPSVR) method inspired by Nonparallel Support Vector Machines (NPSVM).
• Address the limitations of existing TSVR methods by incorporating SRM and enhancing model sparseness.

Main Methods:

• Introduce a regularized term in primal problems, adhering to the SRM principle for direct kernel trick application in nonlinear cases.
• Utilize an ε-insensitive loss function to maintain inherent sparseness, similar to standard Support Vector Regression (SVR).
• Formulate dual problems akin to standard SVR to avoid matrix inversion and employ a Sequential Minimization Optimization (SMO)-type solver for efficient training.

Main Results:

• The proposed NPSVR method demonstrates superior performance compared to existing TSVR methods.
• Achieved enhanced sparseness, improved generalization ability, and greater scalability on diverse datasets.
• Numerical experiments validate the effectiveness of NPSVR in practical applications.

Conclusions:

• NPSVR offers a significant advancement over traditional TSVR by integrating SRM and maintaining sparseness.
• The method's design facilitates efficient training and application to large-scale machine learning problems.
• NPSVR shows strong potential for improving regression tasks requiring robust and sparse models.