Bayesian Variable Selection in Linear Regression in One Pass for Large Data Sets

Area of Science:

• Computational Statistics
• Machine Learning
• Database Systems

Background:

• Bayesian models often rely on Markov Chain Monte Carlo (MCMC) methods for computation.
• MCMC methods require numerous iterations, posing challenges for large datasets.
• Variable selection in linear regression is computationally intensive due to its combinatorial nature.

Purpose of the Study:

• To accelerate Bayesian model computation for variable selection in linear regression.
• To develop an efficient algorithm that overcomes the limitations of traditional MCMC methods.
• To integrate Bayesian variable selection into database management systems.

Main Methods:

• Developed a fast Gibbs sampler algorithm with optimizations for Bayesian variable selection.
• Utilized non-informative and conjugate prior distributions for efficient data summarization.
• Employed sparse binary vectors for efficient matrix projections and hash tables for variable subset probabilities.
• Integrated the algorithm into a database management system (DBMS) using User-Defined Functions and stored procedures.

Main Results:

• The proposed algorithm achieves accurate results comparable to existing methods.
• Demonstrated linear scalability with respect to dataset size.
• Achieved orders-of-magnitude speedup compared to the R package for Bayesian variable selection.
• Showcased efficient parallel data summarization and matrix manipulation within a DBMS.

Conclusions:

• The optimized Gibbs sampler significantly accelerates Bayesian model computation for variable selection.
• Integrating the algorithm into a DBMS enhances performance and scalability.
• This approach offers a practical and efficient solution for variable selection in large-scale Bayesian analyses.