Two-Stage Metropolis-Hastings for Tall Data.

Area of Science:

• Statistics
• Computational Statistics
• Machine Learning

Background:

• Tall data problems pose significant computational challenges for Bayesian classification methods.
• Existing approaches like parallelizing likelihood, subsampling, and consensus Monte Carlo have limitations.
• Efficiently handling large datasets is crucial for accurate Bayesian model inference.

Purpose of the Study:

• To introduce a novel computational method for Bayesian classification in tall data scenarios.
• To reduce the computational cost associated with exact likelihood calculations.
• To enhance the scalability of Bayesian classification models for large datasets.

Main Methods:

• A new two-stage Metropolis-Hastings algorithm is proposed.
• The first stage uses an approximate likelihood for proposal screening.
• The second stage performs full likelihood computation only for accepted proposals.

Main Results:

• The proposed two-stage method significantly reduces computational burden in tall data settings.
• The algorithm is adaptable to existing frameworks like consensus Monte Carlo.
• Successful application demonstrated in logistic regression, hierarchical logistic regression, and Bayesian MARS.

Conclusions:

• The two-stage Metropolis-Hastings algorithm offers an efficient solution for Bayesian classification with tall data.
• This method improves computational feasibility for complex models on large datasets.
• The approach enhances the practical applicability of Bayesian methods in data-intensive fields.