Incremental learning for ν-Support Vector Regression.

Area of Science:

• Machine Learning
• Computational Statistics

Background:

• ν-Support Vector Regression (ν-SVR) is an effective regression algorithm offering control over support vectors and automatic tube width adjustment.
• A key challenge in ν-SVR is the lack of an effective initial solution for incremental learning due to its objective function differing from ν-Support Vector Classification (ν-SVC).

Purpose of the Study:

• To develop an exact and effective incremental ν-SVR learning algorithm.
• To address the challenge of generating an effective initial solution for incremental ν-SVR.

Main Methods:

• Proposed a novel 'initial adjustments' procedure to modify ν-SVC weights based on Karush-Kuhn-Tucker (KKT) conditions, creating a suitable initial solution.
• Integrated this procedure with the Accurate On-line ν-SVC (AONSVM) algorithm's steps to create the Incremental ν-SVR (INSVR) algorithm.
• Provided theoretical analysis confirming the existence of crucial inverse matrices essential for INSVR's operation.

Main Results:

• INSVR effectively converges to the optimal solution by avoiding infeasible updating paths.
• Experiments show INSVR is faster than batch ν-SVR algorithms, even with cold and warm starts.
• Theoretical analysis validates the core components of the INSVR algorithm.

Conclusions:

• The proposed INSVR algorithm provides an effective and exact solution for incremental ν-SVR learning.
• INSVR demonstrates superior performance in terms of speed and convergence compared to traditional batch methods.