Quantized Minimum Error Entropy Criterion.

Area of Science:

• Machine Learning
• Information Theory
• Signal Processing

Background:

• Traditional learning criteria like mean square error are limited in nonlinear and non-Gaussian scenarios.
• Minimum Error Entropy (MEE) and its Information Potential (IP) cost function offer superior performance in these complex data conditions.
• The quadratic computational complexity of IP (O(N^2)) presents a significant bottleneck for large datasets.

Purpose of the Study:

• To develop an efficient method to reduce the computational burden of the Information Potential (IP) cost function.
• To introduce a novel learning criterion, Quantized Minimum Error Entropy (QMEE), for improved computational efficiency.
• To analyze the fundamental properties of the proposed QMEE criterion.

Main Methods:

• An efficient quantization approach was developed to approximate the Information Potential (IP).
• The computational complexity was reduced from O(N^2) to O(MN), where M << N.
• Basic properties of the new Quantized Minimum Error Entropy (QMEE) criterion were theoretically investigated.

Main Results:

• The proposed quantization approach effectively reduces the computational complexity of IP.
• The new QMEE criterion achieves a significant decrease in computational cost, making it suitable for large-scale data.
• Illustrative examples demonstrated the excellent performance of QMEE in linear-in-parameter models.

Conclusions:

• QMEE offers a computationally efficient alternative to traditional IP for MEE-based learning.
• The method addresses the scalability issues associated with IP in machine learning and signal processing.
• QMEE shows promise for applications requiring efficient processing of large, complex datasets.