Entropy-based incremental variational Bayes learning of Gaussian mixtures

Area of Science:

• Machine Learning
• Pattern Recognition
• Statistical Modeling

Background:

• Gaussian mixture models (GMMs) are widely used for density estimation and pattern recognition.
• Optimizing GMM complexity and learning parameters simultaneously is a significant challenge.
• Existing methods often require careful initialization and can be computationally intensive.

Purpose of the Study:

• To develop an unsupervised, incremental learning scheme for GMMs.
• To enable simultaneous model learning and complexity optimization.
• To avoid the need for prior parameter initialization.

Main Methods:

• An entropy-based variational learning scheme is proposed.
• The method employs an incremental approach, starting with a single component.
• New components are added iteratively by splitting the least-fitting kernel based on entropy evaluation.
• Model selection is achieved through efficient variational Bayes optimization.

Main Results:

• The proposed method demonstrates effective learning and complexity optimization for GMMs.
• Experimental results on synthetic and real datasets show superior performance.
• The approach outperforms other state-of-the-art incremental component learning methods.

Conclusions:

• The developed incremental entropy-based variational learning scheme offers an effective solution for GMMs.
• It provides a robust and efficient alternative to existing methods, particularly in unsupervised learning scenarios.
• The method successfully addresses the challenge of simultaneous model learning and complexity optimization.