Kernel-free quadratic surface SVM for conditional probability estimation in imbalanced multi-class classification.

Area of Science:

• Machine Learning
• Pattern Recognition

Background:

• Multi-class classification problems present challenges in accurate probability estimation.
• Existing methods may struggle with imbalanced datasets and computational complexity.

Purpose of the Study:

• To propose a novel probabilistic output classifier, kernel-free quadratic surface support vector machine for conditional probability estimation (CPSQSVM), for multi-class classification.
• To develop a binary classifier (BCPSQSVM) that estimates conditional probability density as a quadratic function.

Main Methods:

• The CPSQSVM utilizes a one-vs-rest (OvR) decomposition strategy combined with the BCPSQSVM.
• The BCPSQSVM formulates a convex quadratic programming problem solvable without kernel functions.
• A block iteration algorithm is designed for the dual problem to handle large constraint sizes.
• Greater weights are assigned to minority samples to address labeling imbalance.

Main Results:

• Theoretical analysis confirms the existence and uniqueness of optimal solutions, reliability, and versatility of CPSQSVM.
• Convergence of the algorithm and an upper bound on the margin parameter are analyzed.
• Numerical experiments on artificial and benchmark datasets validate the method's feasibility and effectiveness.

Conclusions:

• The proposed CPSQSVM offers a robust and efficient solution for multi-class classification problems, particularly those with imbalanced data.
• The kernel-free approach simplifies computation while maintaining high performance in probability estimation.