Kernel-Free Quadratic Surface Regression for Multi-Class Classification.

Area of Science:

• Machine Learning
• Computer Science

Background:

• Multi-class classification presents challenges in developing effective nonlinear classifiers.
• Existing methods may lack generalization ability or have complex optimization problems.

Purpose of the Study:

• Introduce a novel kernel-free nonlinear classifier, Hard Quadratic Surface Least Squares Regression (HQSLSR).
• Propose a softened version (SQSLSR) to improve generalization performance.
• Analyze the theoretical properties and experimental efficacy of the proposed methods.

Main Methods:

• Developed HQSLSR by combining least squares loss with a quadratic kernel-free trick.
• Introduced an ε-dragging technique in SQSLSR to enlarge between-class distances.
• Employed an alteration iteration algorithm to solve the SQSLSR optimization problem.

Main Results:

• HQSLSR offers a convex and unconstrained optimization problem.
• SQSLSR demonstrated improved generalization ability and geometric diversity in regression functions.
• Theoretical analysis confirmed convergence, interpretability, and computational efficiency.
• Experimental results on benchmark datasets showed competitive performance against state-of-the-art classifiers.

Conclusions:

• The proposed HQSLSR and SQSLSR methods provide effective solutions for multi-class classification.
• The ε-dragging technique is beneficial for enhancing classifier generalization.
• These kernel-free approaches offer a promising alternative for nonlinear classification tasks.