Regularized mixture density estimation with an analytical setting of shrinkage intensities

Area of Science:

• Machine Learning
• Statistics
• Data Science

Background:

• Accurate probability density estimation is crucial for many statistical and machine learning tasks.
• Gaussian Mixture Models (GMMs) are widely used for density estimation but can struggle with covariance matrix estimation, especially in high dimensions.
• Existing methods for GMM estimation often face challenges with accuracy and computational efficiency.

Purpose of the Study:

• To propose a novel method for P-variate probability density estimation using a regularized Gaussian mixture model (RGMM).
• To enhance the estimation accuracy of GMM component covariance matrices through a regularization technique.
• To develop an expectation maximization (EM) algorithm for fitting the proposed RGMM.

Main Methods:

• The proposed method utilizes a regularization technique for covariance matrix estimation within a GMM framework.
• An analytical Ledoit-Wolf-type shrinkage estimation is employed for covariance matrices.
• An expectation maximization (EM) algorithm is derived for fitting the regularized GMM (RGMM).

Main Results:

• The RGMM method demonstrates significant improvements in multivariate probability density estimation accuracy.
• Performance comparisons were conducted using both synthetic and real-world datasets.
• The proposed RGMM outperformed recent model-based and variational Bayes approximation methods.

Conclusions:

• The proposed RGMM method offers a superior approach for P-variate probability density estimation.
• The regularization technique effectively enhances the accuracy of GMM covariance matrix estimation.
• RGMM provides a robust and high-performing solution for multivariate density estimation problems.