Genetic-based EM algorithm for learning Gaussian mixture models

Area of Science:

• Machine Learning
• Statistical Modeling
• Computational Statistics

Background:

• Gaussian Mixture Models (GMMs) are widely used for data clustering and density estimation.
• Traditional Expectation-Maximization (EM) algorithm can be sensitive to initialization and may converge to local optima.
• Determining the optimal number of components in GMMs remains a challenge.

Purpose of the Study:

• To propose a novel GA-EM algorithm for learning GMMs from multivariate data.
• To enable automatic selection of the number of GMM components using the Minimum Description Length (MDL) criterion.
• To improve upon the performance of the standard EM algorithm in terms of solution quality and component selection.

Main Methods:

• A hybrid algorithm combining Genetic Algorithms (GA) and the EM algorithm (GA-EM).
• Utilizing GA's population-based stochastic search to explore the solution space more effectively.
• Incorporating an elitist strategy to maintain monotonic convergence properties.
• Employing the MDL criterion for automatic model order selection.

Main Results:

• The GA-EM algorithm demonstrated superior performance compared to the standard EM algorithm on simulated and real datasets.
• Achieved better MDL scores under identical termination conditions.
• More frequently identified the correct number of underlying components in the data.
• Showed reduced sensitivity to initialization, mitigating the local optima problem.

Conclusions:

• The proposed GA-EM algorithm offers a robust and effective approach for learning Gaussian Mixture Models.
• It successfully addresses the limitations of the standard EM algorithm, particularly regarding local optima and model selection.
• The GA-EM algorithm provides a more reliable method for GMM parameter estimation and component determination.