Probability density estimation with tunable kernels using orthogonal forward regression.

Area of Science:

• Statistics
• Machine Learning
• Data Science

Background:

• Probability density function (PDF) estimation is crucial for statistical modeling and machine learning.
• Traditional methods like finite mixture models can suffer from high-dimensional optimization challenges.
• Fixed-kernel models have limitations in generalization and sparsity.

Purpose of the Study:

• To propose a novel generalized or tunable-kernel model for probability density function estimation.
• To enhance model generalization capability and sparsity.
• To avoid complex optimization issues found in conventional models.

Main Methods:

• Utilizing an orthogonal forward regression procedure for kernel determination.
• Minimizing a leave-one-out test criterion to tune kernel centers and covariance matrices.
• Employing multiplicative nonnegative quadratic programming for kernel weight updates, ensuring nonnegativity and unity constraints.

Main Results:

• The tunable-kernel model demonstrates superior generalization and sparsity over fixed-kernel models.
• The model effectively constructs compact and accurate density estimates.
• The weight-updating process contributes to model size reduction.

Conclusions:

• The proposed tunable-kernel model offers an effective and efficient approach to probability density estimation.
• It provides a competitive alternative to existing methods, particularly in scenarios requiring model compactness and accuracy.
• The method successfully addresses limitations of conventional finite mixture and fixed-kernel models.