Gradient-Modulated Cross-Entropy Loss for Robust Learning With Label Noise

Deep neural networks (DNNs) trained with the standard cross-entropy (CE) loss typically achieve strong performance on clean datasets but suffer from severe overfitting when confronted with label noise. To alleviate this issue, numerous robust loss functions have been proposed and shown to improve noise tolerance. However, most existing methods focus predominantly on the target class, overlooking informative class-structural cues embedded in nontarget predictions. In this article, we propose a simple yet effective loss function, named gradient-modulated CE (GMCE), which enhances robustness by incorporating uncertainty-aware gradient modulation across the entire predicted distribution. GMCE comprises two synergistic components: negative-shifted CE (NSCE) loss and uncertainty-aware calibration mechanism (UACM). Specifically, NSCE attenuates the gradient magnitude for highly uncertain predictions by applying a convexity-preserving negative shift to the CE loss, thereby flattening its curvature in low-confidence regions. In parallel, UACM adaptively reweights the loss magnitude based on the entropy and cumulative probability over nontarget classes, assigning greater emphasis to samples with peaked and confident output distributions. Furthermore, we provide a theoretical analysis demonstrating the robustness of GMCE under label noise. Extensive experiments on benchmark datasets with both synthetic and real-world label noise, including MNIST, CIFAR-10, CIFAR-100, WebVision, and Clothing1M, demonstrate that the proposed method consistently outperforms the state-of-the-art approaches.