Consistent high-dimensional Bayesian variable selection via penalized credible regions.

Area of Science:

• Statistics
• Machine Learning
• Computational Statistics

Background:

• High-dimensional data analysis presents challenges in selecting relevant predictors when the number of predictors exceeds the sample size.
• Existing methods like sure screening, forward selection, and penalized regressions have limitations.
• Traditional Bayesian variable selection often relies on computationally intensive Markov Chain Monte Carlo (MCMC) simulations and sensitive prior choices.

Purpose of the Study:

• To develop a more efficient and robust Bayesian variable selection method for high-dimensional regression.
• To address the computational and sensitivity issues associated with existing Bayesian approaches.
• To establish consistent model selection properties in high-dimensional settings.

Main Methods:

• Proposing a conjugate prior on the full model parameters.
• Utilizing sparse solutions within posterior credible regions for variable selection.
• Leveraging closed-form representations of posterior credible regions for computational efficiency.
• Employing existing algorithms for computing sparse solutions.

Main Results:

• The proposed method demonstrates superior performance compared to common techniques in high-dimensional settings, especially when predictors are correlated.
• Consistent model selection is achieved by searching for sparse solutions within joint credible regions.
• Under specific conditions, marginal credible intervals enable consistent selection even when dimensionality grows exponentially with sample size.

Conclusions:

• The novel Bayesian approach effectively performs variable selection in high-dimensional data.
• It avoids common pitfalls of traditional Bayesian variable selection methods, offering improved efficiency and robustness.
• The method provides a computationally tractable and statistically sound alternative for analyzing complex datasets.