A convergent hybrid decomposition algorithm model for SVM training.

Area of Science:

• Machine Learning
• Optimization Algorithms
• Computational Science

Background:

• Training Support Vector Machines (SVMs) involves solving large-scale convex quadratic programming problems.
• Standard methods struggle with massive datasets due to memory constraints for storing the Hessian matrix.
• Decomposition algorithms and caching techniques are used to mitigate these issues but face convergence challenges.

Purpose of the Study:

• To propose a general hybrid algorithm model for Support Vector Machine (SVM) training.
• To combine global convergence properties with flexible utilization of cached Hessian matrix information.
• To address the computational challenges posed by large training datasets in SVMs.

Main Methods:

• Development of a general hybrid algorithm framework for decomposition methods.
• Implementation of a specific algorithm realizing the hybrid model, focusing on a novel caching strategy.
• Utilizing working sets and caching techniques for efficient Hessian matrix management during optimization.

Main Results:

• The proposed hybrid algorithm demonstrates the capability to produce globally convergent sequences of points.
• The flexible caching strategy allows for effective exploitation of stored information without compromising convergence.
• Computational experiments with simple implementations show promising numerical results, indicating the approach's potential.

Conclusions:

• The hybrid algorithm model offers a robust solution for training Support Vector Machines (SVMs) on large datasets.
• The flexible caching strategy enhances computational efficiency while maintaining convergence guarantees.
• The approach shows significant potential for improving the practical application of SVMs in data-intensive scenarios.