Efficient inference in state-space models through adaptive learning in online Monte Carlo expectation maximization

Area of Science:

• Computational Statistics
• Machine Learning
• Financial Modeling

Background:

• Expectation Maximization (EM) estimates parameters for latent variable models.
• Stochastic Approximation EM (SAEM) handles intractable statistics using Monte Carlo methods.
• Batch EM (BEM) and Online EM (OEM) are SAEM variants sensitive to learning rate parameters.

Purpose of the Study:

• To introduce Introspective Online Expectation Maximization (IOEM), an extension of OEM.
• To eliminate the need for manual learning rate selection in SAEM algorithms.
• To enhance the efficiency and applicability of EM-based parameter estimation.

Main Methods:

• Developed the Introspective Online Expectation Maximization (IOEM) algorithm.
• IOEM dynamically adapts the learning rate based on parameter update trends.
• Evaluated IOEM performance against optimal BEM and OEM across multiple models.

Main Results:

• IOEM demonstrates comparable efficiency to optimal BEM and OEM algorithms.
• IOEM's efficiency surpasses BEM/OEM with optimal learning rates in high-dimensional models.
• Successfully applied IOEM to fit financial time series models.

Conclusions:

• IOEM offers a robust and parameter-free alternative to existing SAEM methods.
• The adaptive learning rate mechanism enhances computational efficiency and stability.
• IOEM provides a valuable tool for complex statistical modeling, including financial applications.