A Robust Learning Approach for Regression Models Based on Distributionally Robust Optimization.

Area of Science:

• Machine Learning
• Statistics
• Optimization

Background:

• Linear regression models are sensitive to outliers.
• Adversarially corrupted outliers can significantly degrade model performance.

Purpose of the Study:

• To develop a robustified regression plane estimation method using Distributionally Robust Optimization (DRO).
• To mitigate the impact of outliers in linear regression settings.
• To provide theoretical guarantees and practical insights into regularization.

Main Methods:

• Employing a Distributionally Robust Optimization (DRO) approach.
• Hedging against a family of probability distributions close to the empirical distribution (Wasserstein metric).
• Relaxing the DRO formulation to a convex optimization problem.

Main Results:

• The DRO formulation recovers common regularized regression models.
• New insights into regularization terms and coefficient selection are provided.
• Performance guarantees for prediction and estimation bias are established.
• Demonstrated superiority over existing regression models in extensive numerical results.
• Achieved higher Area Under the ROC Curve (AUC) in outlier detection compared to M-estimation.

Conclusions:

• The proposed DRO approach offers a robust and accurate method for linear regression in the presence of outliers.
• The framework provides a principled way to select regularization parameters.
• The method shows significant promise for both robust regression and outlier detection applications.