parallelMCMCcombine: an R package for bayesian methods for big data and analytics.

Area of Science:

• Statistics
• Computational Statistics
• Data Science

Background:

• Big data analytics face challenges with large datasets exceeding memory and storage limits.
• Traditional Bayesian methods struggle with datasets that are large due to sample size.
• Recent developments focus on partitioning large datasets for Bayesian analysis.

Purpose of the Study:

• Introduce the R package parallelMCMCcombine for handling big data in Bayesian analysis.
• Provide tools to implement and explore methods for combining independent subset posterior samples.
• Demonstrate the utility of these methods for various Bayesian models.

Main Methods:

• Utilizes Bayesian methods that partition big data into subsets.
• Performs independent Bayesian Markov chain Monte Carlo (MCMC) analyses on each subset.
• Combines independent subset posterior samples to estimate the full dataset posterior density.
• The parallelMCMCcombine R package implements four distinct combination techniques.

Main Results:

• Demonstrated effectiveness of subset combination methods for Bayesian logistic regression, Gaussian mixture, and hierarchical models.
• Illustrated the R package's functionality with a Bayesian logistic regression model on simulation data.
• Showcased the package's capabilities using a Bayesian Gamma model on real-world data.

Conclusions:

• The parallelMCMCcombine package offers a practical tool for researchers analyzing large datasets with Bayesian methods.
• The implemented techniques are suitable for models with fixed-dimension parameters in continuous spaces.
• This tool aids exploration of various subset combination approaches, advancing big data Bayesian research.