Negotiating Multicollinearity with Spike-and-Slab Priors

Area of Science:

• Statistics
• Machine Learning

Background:

• Multicollinearity in normal linear models causes unstable estimates.
• This instability complicates variable selection in regression analysis.

Purpose of the Study:

• To demonstrate how spike-and-slab priors improve variable selection under multicollinearity.
• To compare the performance of three expectation-maximization (EM) algorithms for identifying high posterior models.

Main Methods:

• Utilizing spike-and-slab priors to refine the likelihood surface into a multimodal posterior distribution.
• Implementing and comparing three EM algorithms: EMVS (Rockova and George, 2014) and two new variants.
• Analyzing the regions of convergence for these algorithms in the presence of multimodal posteriors.

Main Results:

• Spike-and-slab priors effectively allocate likelihood information to subset model modes.
• Deterministic annealing versions of EMVS significantly reduce posterior multimodality.
• The study compares the convergence properties of the three EM algorithms.

Conclusions:

• Spike-and-slab priors offer a robust solution for variable selection in the presence of multicollinearity.
• EM algorithms, particularly deterministic annealing variants, are effective tools for navigating complex posterior distributions.
• The findings provide practical methods for enhancing regression model stability and interpretability.