Efficient EM Estimation for the Pogit Model via Polya-Gamma Augmentation.

Area of Science:

• Statistics
• Econometrics
• Computational Statistics

Background:

• The Poisson-logistic (pogit) model is essential for analyzing count data with latent intensities.
• Existing estimation methods for pogit models struggle with large datasets, limiting their practical application.
• Applications include under-reporting correction and share-of-wallet estimation.

Purpose of the Study:

• To develop a computationally efficient and scalable algorithm for estimating the standard pogit model.
• To address the limitations of existing methods in handling large-scale datasets.
• To provide a competitive alternative for large-scale pogit estimation.

Main Methods:

• Proposed a novel Expectation-Maximization (EM) algorithm utilizing Polya-Gamma data augmentation.
• The algorithm yields a conditionally Gaussian complete-data likelihood with closed-form EM-updates.
• Incorporated computational enhancements like quasi-Newton acceleration and mini-batch implementations for efficiency.

Main Results:

• The new EM algorithm demonstrates low per-iteration cost, enabling efficient inference on millions of observations.
• Simulation studies and real-data applications confirm substantial computational improvements.
• Statistical accuracy is maintained compared to existing methods.

Conclusions:

• The proposed EM algorithm offers a scalable and computationally efficient solution for large-scale pogit model estimation.
• It provides a competitive alternative to direct maximum-likelihood optimization routines.
• The method enhances the applicability of pogit models to big data scenarios.