Cluster number selection for a small set of samples using the Bayesian Ying-Yang model.

Area of Science:

• Computational statistics
• Machine learning
• Data mining

Background:

• Determining the optimal number of clusters is a fundamental challenge in cluster analysis.
• Existing methods may be computationally intensive or less accurate for small sample sizes.

Purpose of the Study:

• To introduce a robust method for determining the number of clusters in small datasets.
• To refine the Bayesian-Kullback Ying-Yang (BYY) model selection criterion for practical application.

Main Methods:

• Application of the Bayesian-Kullback Ying-Yang (BYY) model selection criterion.
• Derivation of a new equation for smoothing parameter estimation under a second-order approximation.
• Development of a gradient descent smoothing parameter estimation approach.

Main Results:

• The proposed gradient descent method effectively estimates the smoothing parameter.
• This approach avoids complex integration procedures.
• The method achieves optimal results comparable to existing techniques.

Conclusions:

• The Bayesian-Kullback Ying-Yang (BYY) criterion, enhanced with gradient descent smoothing parameter estimation, provides an efficient solution for determining cluster numbers.
• This method is particularly valuable for analyzing small datasets in cluster analysis.