Simple proof of convergence of the SMO algorithm for different SVM variants

Area of Science:

• Machine Learning
• Computational Statistics

Background:

• The Sequential Minimum Optimization (SMO) algorithm is crucial for training Support Vector Machines (SVMs).
• Existing convergence proofs for SMO can be complex and lengthy.

Purpose of the Study:

• To provide a novel, more accessible proof for the asymptotic convergence of the SMO algorithm.
• To demonstrate the broad applicability of the proof across different SVM formulations.

Main Methods:

• A new proof methodology is introduced, drawing inspiration from Gilbert's work on minimum norm problems.
• The proof is developed for a general problem formulation encompassing Support Vector Classification (SVC) and Support Vector Regression (SVR).

Main Results:

• A self-contained, shorter, and simpler proof of SMO convergence is established.
• The proof is validated for both most violating pair and second order rules.
• The generalized proof extends to other SVM variants like -SVC and one-class SVMs with minimal modifications.

Conclusions:

• The novel convergence proof simplifies understanding and verification of SMO algorithm performance.
• This work broadens the theoretical foundation for various SVM applications, including classification and regression.