A new iterative initialization of EM algorithm for Gaussian mixture models.

Area of Science:

• Machine Learning
• Statistical Modeling
• Data Analysis

Background:

• The Expectation Maximization (EM) algorithm is widely used for Gaussian Mixture Model (GMM) parameter estimation.
• However, EM is prone to local optima and sensitive to initial parameter values, hindering its effectiveness.

Purpose of the Study:

• To introduce a novel iterative method for EM initialization, termed MRIPEM (Multiple Restarts, Iterations, and Clustering).
• To enhance the robustness and accuracy of GMM parameter estimation by overcoming the limitations of traditional EM initialization.

Main Methods:

• MRIPEM utilizes multiple restarts, iterative refinement, and clustering techniques.
• Initial GMM parameters are derived from sample mean vectors and covariance matrices.
• Optimal feature vectors are selected using maximum Mahalanobis distance for refined clustering and parameter updates.

Main Results:

• MRIPEM was compared against two other popular EM initialization methods using simulated and real-world datasets.
• The MRIPEM algorithm demonstrated comparable performance in high-dimensional and high-overlap data scenarios.
• MRIPEM significantly outperformed existing methods in low-dimensional and low-overlap data scenarios.

Conclusions:

• The proposed MRIPEM algorithm offers a robust and effective solution for initializing Gaussian Mixture Models.
• MRIPEM addresses the critical issues of local optima and initial value sensitivity in EM algorithms.
• This method shows particular promise for applications involving complex or challenging datasets.