Hard thresholding regression

Area of Science:

• Statistics
• Machine Learning
• Econometrics

Background:

• High-dimensional data presents challenges for traditional regression.
• Sparse linear regression models are crucial for identifying relevant predictors.

Purpose of the Study:

• To propose a novel method, hard thresholding regression (HTR), for estimating high-dimensional sparse linear regression models.
• To develop a two-stage convex algorithm that approximates ℓ0-penalized regression.

Main Methods:

• A two-stage convex algorithm is employed.
• The first stage computes an initial estimator.
• The second stage refines the estimate by leveraging information from the initial stage.

Main Results:

• The proposed HTR estimator demonstrates strong oracle properties.
• The method is robust across a wide range of regularization parameters.
• Numerical simulations and a real-world data example validate the methodology.

Conclusions:

• HTR provides an effective approach for high-dimensional sparse linear regression.
• The two-stage algorithm offers theoretical guarantees and practical performance.
• The method is well-supported by empirical evidence.