Efficient Regularized Regression with L₀ Penalty for Variable Selection and Network Construction

Area of Science:

• Bioinformatics
• Computational Biology
• Statistical Learning

Background:

• High-dimensional big data analysis requires effective variable selection methods.
• L0 regularized regression directly penalizes the number of non-zero features but is computationally challenging (NP-hard).

Purpose of the Study:

• To propose efficient algorithms, L0EM and DL0EM, for approximating L0 optimization in regression.
• To provide solutions for L_p regression problems (p in [0,2]), including lasso and elastic net.
• To enable accurate variable selection and network construction in high-dimensional biological data.

Main Methods:

• Developed efficient EM (L0EM) and dual L0EM (DL0EM) algorithms to directly approximate L0 optimization.
• L0EM is efficient for large sample sizes; DL0EM is efficient for high-dimensional data (n << m).
• Regularized parameter lambda determined via cross-validation, AIC, or BIC.

Main Results:

• L0 methods demonstrated superior performance compared to lasso, SCAD, and MC+ in simulations and genomic data analysis.
• L0 with AIC or BIC achieved performance comparable to computationally intensive cross-validation.
• Proposed algorithms efficiently identified non-zero variables with reduced bias.

Conclusions:

• L0EM and DL0EM provide efficient and effective solutions for variable selection in high-dimensional regression.
• These methods facilitate the construction of biologically relevant networks from big data.
• The algorithms offer a robust approach for various L_p regularization problems.