CALIBRATING NON-CONVEX PENALIZED REGRESSION IN ULTRA-HIGH DIMENSION

Area of Science:

• Statistics
• Machine Learning
• Data Science

Background:

• High-dimensional non-convex penalized regression presents challenges in identifying the oracle estimator due to multiple local minima.
• Existing methods struggle with non-unique solution paths and estimating optimal tuning parameters.

Purpose of the Study:

• To develop a robust method for identifying the oracle estimator in high-dimensional non-convex penalized regression.
• To address the non-uniqueness of solution paths and the difficulty in selecting optimal tuning parameters.

Main Methods:

• A calibrated Convex Concave Procedure Programming (CCCP) algorithm is proposed to generate a consistent solution path.
• A high-dimensional Bayesian Information Criterion (BIC) is developed for optimal tuning parameter selection.
• Theoretical guarantees are established for sub-Gaussian error distributions in ultra-high dimensions.

Main Results:

• The calibrated CCCP algorithm consistently generates a solution path containing the oracle estimator with high probability.
• The proposed high-dimensional BIC effectively selects the optimal tuning parameter, asymptotically identifying the oracle estimator.
• Monte Carlo simulations validate the effectiveness of the combined approach.

Conclusions:

• The calibrated CCCP algorithm and high-dimensional BIC offer a powerful solution for high-dimensional non-convex penalized regression.
• This method reliably identifies the underlying sparsity pattern in high-dimensional data analysis.
• The findings advance the theoretical understanding and practical application of penalized regression in ultra-high dimensions.