Probability Density Rank-Based Quantization for Convex Universal Learning Machines.

Area of Science:

• Machine Learning
• Optimization
• Data Science

Background:

• Convex universal learning machines (CULMs) are powerful for complex system modeling but suffer from high computational complexity due to large hidden layer dimensions.
• Efficient data distribution handling is crucial for the performance of learning machines like CULMs.

Purpose of the Study:

• To propose an efficient quantization method, Probability density Rank-based Quantization (PRQ), to decrease the computational complexity of CULMs.
• To maintain data distribution similarity while reducing computational cost and improving practical applicability of CULMs.

Main Methods:

• Developed Probability density Rank-based Quantization (PRQ) by ranking data based on estimated probability densities and selecting equally spaced subsets.
• Applied PRQ to kernel ridge regression (KRR) and random Fourier feature recursive least squares (RFF-RLS), key CULM algorithms.
• Utilized kd-tree for efficient computation and outlier exclusion, ensuring deterministic quantization results.

Main Results:

• PRQ effectively reduces computational complexity in CULMs.
• The method preserves data distribution similarity between the codebook and dataset.
• Experimental results on benchmark datasets demonstrate satisfactory performance compared to state-of-the-art methods.

Conclusions:

• PRQ offers a significant improvement in the computational efficiency of CULMs.
• The method provides deterministic results, handles outliers, and avoids excessive codebook borders, enhancing practical usability.
• PRQ represents a valuable advancement for applying CULMs to complex real-world problems.