Noise Level Estimation for Model Selection in Kernel PCA Denoising.

Area of Science:

• Machine Learning
• Unsupervised Learning
• Computational Statistics

Background:

• Unsupervised learning models, such as kernel principal component analysis (kPCA), face challenges in optimal parameter selection.
• Key parameters include the number of components, kernel type, and kernel parameters, which significantly impact model performance.

Purpose of the Study:

• To introduce a new model selection criterion, the Model selection based on Distance Distributions (MDDs), for unsupervised learning.
• To enable automatic selection of the number of components and radial basis function kernel parameters (e.g., σ(2)) in kPCA.
• To enhance the scalability of kPCA for large datasets.

Main Methods:

• Developed an MDD criterion based on spectral comparison of information and noise.
• Estimated noise content using statistical moments of distance distributions in the original dataset.
• Proposed modifications to the fixed-size method and employed incomplete Cholesky factorization for large-scale kPCA.

Main Results:

• The MDD criterion effectively identifies optimal model parameters by distinguishing between signal and noise.
• The proposed methods, including MDD, improved fixed-size kPCA, and incomplete Cholesky factorization, demonstrated superior performance.
• Evaluated performance on both synthetic and real-world datasets, showing advantages over existing algorithms.

Conclusions:

• The MDD criterion offers a robust approach for parameter selection in kPCA, enhancing model interpretability and performance.
• The integration of MDD with scalable kPCA techniques provides an efficient solution for large-scale unsupervised learning tasks.
• The presented methods significantly outperform current algorithms in terms of accuracy and efficiency for kPCA.