EM vs MM: A Case Study.

Area of Science:

• Statistics
• Optimization Methods

Background:

• The Expectation-Maximization (EM) algorithm is a widely used statistical optimization method.
• EM is a special case of the more general Minorization-Maximization (MM) principle.
• Both algorithms iteratively optimize objective functions using surrogate functions.

Purpose of the Study:

• To compare the Expectation-Maximization (EM) and Minorization-Maximization (MM) algorithms.
• To derive and analyze algorithms for Dirichlet-Multinomial distribution parameter estimation.
• To introduce and evaluate an EM-MM hybrid algorithm.

Main Methods:

• Construction of both EM and MM algorithms for Dirichlet-Multinomial parameter estimation.
• Derivation of a novel EM-MM hybrid algorithm.
• Theoretical analysis of local convergence rates from the MM perspective.
• Numerical comparison of EM, MM, and hybrid algorithms.

Main Results:

• EM and MM derivations yield distinct algorithms for the Dirichlet-Multinomial distribution.
• The EM algorithm converges quickly but requires solving a complex maximization problem.
• The MM algorithm has simple updates but converges slowly.
• The EM-MM hybrid algorithm demonstrates faster convergence than MM in specific scenarios.

Conclusions:

• The choice between EM and MM depends on the trade-off between convergence speed and computational complexity.
• The proposed EM-MM hybrid algorithm offers a potentially advantageous alternative.
• Theoretical and numerical analyses provide insights into the algorithms' performance characteristics.