Variable selection for partially linear models via Bayesian subset modeling with diffusing prior.

Area of Science:

• Statistics
• Econometrics
• Machine Learning

Background:

• Partially linear models (PLM) with ultrahigh dimensional covariates present challenges for variable selection.
• Existing two-step methods using partial residuals are susceptible to estimation errors and multicollinearity.
• There is a need for robust variable selection methods that handle high-dimensional and correlated predictors efficiently.

Purpose of the Study:

• To propose a novel one-step Bayesian variable selection approach for PLM.
• To address challenges posed by ultrahigh dimensional covariates and multicollinearity simultaneously.
• To enhance model selection consistency and performance compared to existing methods.

Main Methods:

• A one-step Bayesian variable selection method for PLM is developed.
• The method utilizes a difference-based approach to mitigate estimation impacts from the nonparametric component.
• Bayesian subset modeling with diffusing prior (BSM-DP) is incorporated for the linear component.
• Gibbs sampling is employed for estimation.

Main Results:

• The proposed method achieves model selection consistency, even with covariates growing exponentially with sample size.
• It demonstrates superior performance in the presence of highly correlated predictors.
• Asymptotic analysis shows the posterior probability of selecting the true model converges to one.
• Simulation studies validate the method's efficiency and theoretical underpinnings.

Conclusions:

• The novel Bayesian approach offers a robust and efficient solution for variable selection in ultrahigh dimensional PLM.
• It effectively handles multicollinearity, a common issue in complex datasets.
• The method provides a significant advancement over existing two-step procedures, with practical applications demonstrated in supermarket data analysis.