An SMO algorithm for the potential support vector machine.

Area of Science:

• Machine Learning
• Optimization Algorithms
• Computational Science

Background:

• Standard support vector machines (SVMs) have limitations in handling arbitrary dyadic datasets.
• The potential support vector machine (P-SVM) offers broader applicability but requires efficient optimization methods.
• Sequential Minimal Optimization (SMO) is a common technique for solving SVM dual optimization problems.

Purpose of the Study:

• To develop and evaluate a fast sequential minimal optimization (SMO) procedure for the potential support vector machine (P-SVM) dual optimization problem.
• To introduce and assess heuristics for improving SMO performance, including block optimization and annealing of the regularization parameter.
• To compare the computational efficiency and performance of the enhanced P-SVM against standard SVM implementations.

Main Methods:

• Developed a sequential minimal optimization (SMO) procedure, including single and dual SMO variants, for P-SVM.
• Implemented efficient selection procedures for Lagrange multipliers.
• Incorporated heuristics: block optimization and annealing of the regularization parameter (epsilon).
• Benchmarked against libSVM's epsilon-SVR and C-SVC for comparable problems.

Main Results:

• The dual SMO procedure, enhanced with block optimization and annealing, demonstrated efficient computation times.
• P-SVM is applicable to arbitrary dyadic datasets, unlike standard SVMs.
• For problems solvable by standard SVMs, P-SVM computation time was comparable or slightly higher.
• P-SVM consistently yielded a significantly smaller number of support vectors for equivalent generalization performance.

Conclusions:

• The proposed dual SMO with block optimization and annealing is an efficient method for P-SVM optimization.
• P-SVM offers a viable alternative to standard SVMs, particularly for dyadic data, with benefits in support vector reduction.
• The P-SVM approach shows promise for improving model sparsity and generalization in machine learning applications.