Improvement of KLMS algorithm based on attention mechanism

This paper proposes an attention-augmented kernel adaptive filtering framework that enhances the Kernel Least Mean Square (KLMS) algorithm by explicitly incorporating data-dependent attention mechanisms into its update structure. By reinterpreting the KLMS recursion from an attention perspective, an additive attention-based variant, termed AttKLMS, is developed to dynamically regulate the contribution of historical samples during online learning. This design effectively combines the nonlinear modeling capability of kernel methods with the adaptive focusing property of attention, leading to improved convergence behavior and robustness under noisy conditions. To further address the computational burden and dimensionality challenges encountered in high-dimensional scenarios, a self-attention-based extension, referred to as Self-AttKLMS, is introduced as a learnable front-end representation module. Unlike fixed dimensionality reduction techniques, self-attention enables instance-wise and data-adaptive feature aggregation prior to kernel evaluation, thereby reducing model complexity while preserving essential information for filtering. Extensive experiments conducted on both one-dimensional chaotic time series (Mackey-Glass and Lorenz) and two-dimensional millimeter-wave imaging datasets demonstrate that the proposed methods consistently outperform conventional KLMS variants in terms of prediction accuracy, convergence stability, and noise robustness. The results further indicate that the proposed attention-augmented kernel adaptive filtering framework provides an effective and computationally efficient alternative to existing kernel-based and deep learning approaches in online sequence prediction tasks.