Multiclass Classification and Feature Selection Based on Least Squares Regression with Large Margin.

Area of Science:

• Machine Learning
• Statistical Analysis
• Pattern Recognition

Background:

• Least Squares Regression (LSR) is a foundational technique in feature learning but often neglects local data structures.
• Traditional LSR struggles to precisely reflect classification ability due to its loss function limitations.
• Existing methods attempt to preserve local information using orthogonal constraints and improve classification via large margin concepts.

Purpose of the Study:

• To propose a novel algorithm, Orthogonal Least Squares Regression with Large Margin (OLSLM), for multiclass classification.
• To simultaneously learn regression targets and an orthogonal transformation matrix.
• To enhance classification accuracy with a large margin and preserve local data structure in a subspace.

Main Methods:

• Developed OLSLM, integrating large margin and orthogonal constraints.
• Employed an efficient iterative optimization method for solving constraints, demonstrating theoretical and practical convergence.
• Applied the large margin constraint for sparse learning and feature selection using joint L1-norm minimization.

Main Results:

• The proposed OLSLM model ensures correct classification with a larger margin compared to conventional LSR.
• OLSLM effectively preserves local data structure information within the learned subspace.
• Experimental results show superior performance of OLSLM over state-of-the-art methods on diverse real-world datasets.

Conclusions:

• OLSLM offers a robust approach for multiclass classification by effectively combining large margin and orthogonal constraints.
• The method demonstrates significant improvements in both classification accuracy and local structure preservation.
• OLSLM provides an effective strategy for feature selection, contributing to more interpretable and efficient models.