Generalized and Robust Least Squares Regression.

Area of Science:

• Machine Learning
• Data Science
• Statistical Modeling

Background:

• Least Squares Regression (LSR) is widely used for data analysis but is sensitive to noise.
• Existing methods using sparse representation and L1 regularization for feature selection inherit LSR's noise sensitivity.
• Noise and outliers significantly degrade the performance of classification and feature selection algorithms.

Purpose of the Study:

• To propose a Generalized and Robust Least Squares Regression (GRLSR) method.
• To enhance the robustness of classification and feature selection against noise and outliers.
• To develop an efficient iterative algorithm for solving the non-convex minimization problem.

Main Methods:

• Introduced GRLSR with an arbitrary concave loss function and L2,p-norm regularization.
• Developed an iterative algorithm incorporating sample-specific weights to suppress noise effects.
• Weights are automatically assigned based on sample error, down-weighting noisy data points.

Main Results:

• The proposed GRLSR method demonstrates superior performance on corrupted datasets.
• The iterative algorithm efficiently handles the non-convex minimization problem.
• The weighting mechanism effectively mitigates the impact of noise and outliers.

Conclusions:

• GRLSR offers a robust alternative to traditional LSR for classification and feature selection.
• The method provides improved accuracy and reliability in the presence of noisy data.
• The framework is versatile, with four specific formulations proposed to illustrate its capabilities.