Adaptive penalization in high-dimensional regression and classification with external covariates using variational...

Area of Science:

• Statistics
• Machine Learning
• Bioinformatics

Background:

• Penalized regression methods like Lasso and ridge regression are standard for high-dimensional data.
• The relative strength of penalization is often implicitly determined by predictor scales, ignoring valuable external information.
• Existing methods do not fully utilize available covariate information to adapt penalization strategies.

Purpose of the Study:

• To develop a data-driven method for adapting penalized regression by incorporating external covariates.
• To differentially penalize feature groups based on covariates and their information content.
• To enhance prediction performance and model interpretability in high-dimensional settings.

Main Methods:

• A Bayesian approach is employed to combine shrinkage and feature selection.
• The method introduces a scalable optimization scheme for penalized regression.
• Feature groups are defined by external covariates, allowing for adaptive penalization strengths.

Main Results:

• Simulations show accurate recovery of true effect sizes and sparsity patterns within feature groups.
• The proposed method demonstrates improved prediction performance when groups have varying dynamic ranges.
• Application to high-throughput biology data allows re-weighting feature groups from different assays.

Conclusions:

• The novel penalized regression method effectively utilizes external covariates for adaptive penalization.
• This approach extends the applicability of penalized regression, enhances interpretability, and can boost prediction accuracy.
• The method is particularly beneficial in fields like high-throughput biology where diverse data sources are available.