A Variational Maximization-Maximization Algorithm for Generalized Linear Mixed Models with Crossed Random Effects

Area of Science:

• Statistics
• Machine Learning
• Psychometrics

Background:

• Generalized linear mixed models (GLMMs) are widely used but estimating them with crossed random effects is computationally challenging.
• Existing methods like Laplace approximation struggle with high-dimensional integrals and accuracy in small samples.

Purpose of the Study:

• To develop a novel variational maximization-maximization algorithm for approximate maximum likelihood estimation of GLMMs with crossed random effects.
• To enhance computational efficiency and accuracy compared to existing methods, particularly under small sample conditions.

Main Methods:

• A factorized variational approximation of the latent variable distribution is employed to create a lower bound of the log marginal likelihood.
• This lower bound is maximized with respect to both the factorized distributions and model parameters.
• Adaptive Gauss-Hermite quadrature is integrated to further improve computational efficiency, reducing a high-dimensional integration to a two-dimensional problem.

Main Results:

• The proposed variational algorithm effectively handles GLMMs with crossed random effects.
• Numerical studies demonstrate superior performance over the Laplace approximation, especially under small sample size conditions.
• The method successfully transforms intractable high-dimensional integration into a manageable two-dimensional problem.

Conclusions:

• The developed variational maximization-maximization algorithm offers a more accurate and efficient approach for estimating GLMMs with crossed random effects.
• This method provides a valuable alternative to the Laplace approximation, particularly in challenging small sample data scenarios.
• The integration of adaptive Gauss-Hermite quadrature significantly boosts computational efficiency.