Linear Maximum Margin Classifier for Learning from Uncertain Data.

Area of Science:

• Machine Learning
• Pattern Recognition
• Statistical Learning Theory

Background:

• Traditional Support Vector Machines (SVMs) assume precise data points.
• Real-world data often contains inherent uncertainty, impacting classifier performance.
• Modeling this uncertainty is crucial for robust classification.

Purpose of the Study:

• To develop a maximum margin classifier that explicitly models uncertainty in training data.
• To reformulate the SVM framework to incorporate multi-dimensional Gaussian distributions for each training example.
• To introduce the SVM with Gaussian Sample Uncertainty (SVM-GSU) classifier.

Main Methods:

• Representing each training example as a multi-dimensional Gaussian distribution (mean and covariance matrix).
• Defining a cost function as the expected value of the classical SVM cost over these distributions.
• Solving the resulting convex optimization problem using stochastic gradient descent in primal form.

Main Results:

• The proposed formulation approximates classical SVMs for low-variance isotropic Gaussian inputs.
• The SVM-GSU classifier was tested on synthetic and five diverse public datasets (MNIST, WDBC, DEAP, TV News, TRECVID MED).
• Experimental results demonstrated the effectiveness of the SVM-GSU method.

Conclusions:

• The SVM-GSU effectively handles classification problems with uncertain data inputs.
• The method provides a robust alternative to traditional SVMs when data uncertainty is present.
• The stochastic gradient descent approach ensures efficient solution of the optimization problem.