Mean-square convergence analysis of ADALINE training with minimum error entropy criterion

Area of Science:

• Machine Learning
• Information Theory
• Signal Processing

Background:

• The Minimum Error Entropy (MEE) criterion is an information-theoretic approach for supervised learning.
• MEE presents a non-quadratic, non-convex performance surface, complicating theoretical analysis, especially for Adaptive Linear Neuron (ADALINE) training.
• Traditional mean-square error criteria have limitations that MEE aims to address.

Purpose of the Study:

• To develop a unified approach for analyzing the mean-square convergence of ADALINE training using the MEE criterion.
• To provide a theoretical framework for understanding the stability and performance of MEE-based ADALINE algorithms.
• To bridge the gap between MEE's theoretical complexity and its practical application in adaptive systems.

Main Methods:

• Formulating the weight update equation in a block-data format.
• Utilizing a block version of the energy conservation relation for analysis.
• Applying several assumptions to derive convergence properties.

Main Results:

• The study successfully performed mean-square convergence analysis for ADALINE training under the MEE criterion.
• Analysis covered mean-square stability, transient behavior (evolution), and steady-state performance.
• Simulation results demonstrated strong agreement with the theoretical predictions.

Conclusions:

• The developed unified approach provides a robust method for analyzing MEE-based ADALINE training.
• The theoretical framework accurately predicts the convergence behavior of these adaptive algorithms.
• This work validates the applicability of MEE in supervised learning systems requiring robust convergence analysis.