Fast maximum likelihood estimation for general hierarchical models.

Area of Science:

• Statistics
• Computational Science
• Applied Mathematics

Background:

• Hierarchical statistical models are crucial for analyzing complex data structures in applied sciences.
• Existing maximum likelihood estimation methods, like Monte Carlo Expectation Maximization (MCEM), face challenges with efficiency and generality due to Monte Carlo integration over latent variables.

Purpose of the Study:

• To introduce and evaluate novel methods for efficient estimation in hierarchical statistical models.
• To address the limitations of current Monte Carlo integration techniques for complex hierarchical data.

Main Methods:

• Developed Hierarchical Model Stochastic Gradient Descent (HMSGD) by connecting sampling-stepping iterations to stochastic gradient descent.
• Incorporated efficient, adaptive step-size algorithms, including a one-dimensional sampling-based greedy line search, to enhance HMSGD performance.
• Implemented and tested methods on diverse models: Gamma-Poisson mixture, generalized linear mixed models (GLMMs), and a large-scale ecological occupancy model.

Main Results:

• HMSGD methods demonstrated faster convergence compared to commonly used estimation techniques.
• The accelerated HMSGD approaches proved robust across various model complexities and MCMC sample sizes.
• Numerical experiments confirmed the practical efficiency gains of the proposed methods.

Conclusions:

• Accelerated HMSGD provides a more efficient and robust framework for statistical inference in hierarchical models.
• The developed methods offer significant improvements over existing techniques for complex data analysis in applied sciences.
• HMSGD represents a promising advancement for handling large-scale hierarchical statistical modeling challenges.