Broken adaptive ridge regression and its asymptotic properties

Area of Science:

• Statistics
• Machine Learning
• Econometrics

Background:

• Sparse linear regression is crucial for analyzing high-dimensional datasets.
• Existing oracle variable selection methods lack a grouping effect for correlated covariates.
• The ridge estimator is a common starting point for penalized regression.

Purpose of the Study:

• To introduce and analyze the Broken Adaptive Ridge (BAR) estimator.
• To demonstrate the theoretical properties of BAR, including consistency and the oracle property.
• To investigate the grouping effect of BAR and its performance in high-dimensional settings.

Main Methods:

• Developing an L0-based iteratively reweighted L2 penalization algorithm.
• Utilizing the ridge estimator as an initial value for the BAR estimator.
• Combining BAR with a sparsity-restricted least squares estimator for a two-stage approach.

Main Results:

• The BAR estimator is consistent for variable selection.
• The BAR estimator exhibits the oracle property for parameter estimation.
• BAR demonstrates a natural grouping effect for highly correlated covariates.
• The two-stage method maintains selection and estimation consistency in high-dimensional settings.

Conclusions:

• The BAR estimator offers a robust solution for sparse linear regression with correlated predictors.
• BAR provides a desirable grouping effect, enhancing interpretability.
• The proposed two-stage method extends BAR's utility to ultrahigh-dimensional scenarios.