Retargeted Least Squares Regression Algorithm.

Area of Science:

• Machine Learning
• Computer Science
• Statistical Modeling

Background:

• Multicategory classification is a fundamental task in machine learning.
• Traditional methods like least squares regression (LSR) have limitations in accuracy and efficiency for multiclass problems.
• Existing discriminative LSR models often require training multiple binary classifiers.

Purpose of the Study:

• To present a novel framework, retargeted least squares regression (ReLSR), for multicategory classification.
• To develop a single, compact model that directly learns regression targets from data.
• To enhance classification accuracy and error measurement compared to existing LSR approaches.

Main Methods:

• ReLSR directly learns regression targets from data, deviating from traditional zero-one matrices.
• A learned target matrix enforces a large margin constraint for correct classification.
• The convex optimization problem is solved efficiently using an alternating regression and retargeting procedure.

Main Results:

• ReLSR demonstrates significantly higher accuracy in measuring classification error compared to traditional LSR and discriminative LSR models.
• The proposed method is a single, compact model, eliminating the need for independent binary classifiers.
• Experimental evaluations across various databases validate the effectiveness of the ReLSR framework.

Conclusions:

• ReLSR provides a valid and effective framework for multicategory classification.
• The method offers superior accuracy and a more streamlined approach than existing regression-based classification techniques.
• ReLSR represents a promising advancement in the field of multiclass classification algorithms.