Robust model selection using the out-of-bag bootstrap in linear regression

Area of Science:

• Statistics
• Machine Learning

Background:

• Outliers significantly impact linear model selection.
• Existing methods are sensitive to outlying observations.

Purpose of the Study:

• To develop a robust model selection technique for linear regression in the presence of outliers.
• To improve the reliability of linear model selection when data contains unusual values.

Main Methods:

• A novel model selection criterion combining robust conditional expected prediction loss and a robust goodness-of-fit with a penalty term.
• Estimation of conditional expected prediction loss using an out-of-bag stratified bootstrap approach.
• Utilization of the robust MM-estimator and a bounded loss function to mitigate outlier effects.

Main Results:

• The proposed method demonstrates consistent and satisfactory performance in the presence of both response and covariate outliers.
• Simultaneous minimization of penalized loss and conditional expected prediction loss proved more effective than separate minimization.
• Stratified bootstrap ensures representative samples even with outliers.

Conclusions:

• The developed bootstrap model selection procedure offers a reliable solution for linear regression with outlier-contaminated data.
• This robust approach enhances the accuracy and stability of model selection in practical applications.
• The method effectively addresses the challenges posed by outlying observations in statistical modeling.