Maximum likelihood model selection for 1-norm soft margin SVMs with multiple parameters.

Area of Science:

• Machine Learning
• Computational Statistics
• Pattern Recognition

Background:

• Hyperparameter tuning in Support Vector Machines (SVMs) is complex, especially with flexible kernels and limited data.
• Model selection for SVMs requires robust methods to prevent overfitting and ensure generalization.

Purpose of the Study:

• To present a unified framework for regularized model selection of 1-norm soft margin SVMs for binary classification.
• To develop an efficient gradient-ascent method for optimizing SVM hyperparameters using a likelihood function.

Main Methods:

• Utilized a gradient-ascent approach on a likelihood function derived from logistic regression for hyperparameter optimization.
• Incorporated prior distributions over hyperparameters to mitigate overfitting issues.
• Applied the framework to adapt multiple kernel parameters in SVM models.

Main Results:

• The proposed gradient-based optimization effectively adapted multiple kernel parameters.
• The framework demonstrated superior performance compared to four contemporary state-of-the-art methods.
• The method proved efficient in computing the likelihood function for hyperparameter estimation.

Conclusions:

• The developed framework offers a coherent and effective solution for regularized model selection in SVMs, particularly under data scarcity.
• Gradient-based optimization of the likelihood function is a viable strategy for adapting SVM hyperparameters and improving model performance.