Scalable Bayesian Variable Selection Using Nonlocal Prior Densities in Ultrahigh-dimensional Settings.

Area of Science:

• Statistics
• Machine Learning
• High-Dimensional Data Analysis

Background:

• Variable selection is crucial in high-dimensional statistical modeling.
• Existing Bayesian and frequentist methods face challenges in ultrahigh dimensions.
• Nonlocal prior densities offer an alternative for Bayesian model selection.

Purpose of the Study:

• To extend Bayesian model selection with nonlocal priors to ultrahigh dimensions.
• To compare these methods against established variable selection techniques.
• To develop an efficient algorithm for exploring large model spaces.

Main Methods:

• Bayesian model selection using nonlocal alternative prior densities.
• Comparison with g-priors, reciprocal lasso, adaptive lasso, SCAD, and MCP using precision-recall curves.
• Theoretical analysis of consistency properties for linear models.
• Development and application of the Simplified Shotgun Stochastic Search with Screening (S5) algorithm.

Main Results:

• Bayesian nonlocal prior methods are competitive with other procedures in simulations.
• Nonlocal procedures demonstrate consistency in linear models even as covariates increase sub-exponentially.
• Zellner's g-prior is competitive but induces a more dispersed model space posterior.
• The S5 algorithm significantly reduces computation time without compromising model space exploration.

Conclusions:

• Bayesian variable selection with nonlocal priors is a robust approach for ultrahigh-dimensional data.
• The S5 algorithm provides an efficient computational solution for large-scale Bayesian variable selection.
• The proposed methods and algorithm are available in the R package BayesS5.