Minimizing Time Derivative of Loss for Efficient Generalization Enhancement With Applications to Nickel-Cobalt Alloy...

Deep neural networks (DNNs) achieve impressive performance, yet their effectiveness in complex and real-world settings remains limited by insufficient generalization. Recent studies analyze the geometry of the loss landscape, particularly identifying flat minima, to improve the generalization. To address this issue, this study applies neural dynamics and connects the time derivative of the gradient with flat minima. It then proposes a gradient-directed optimization method that guides parameters of neural networks toward flat minima during the training. Extensive experiments show that the proposed method enhances generalization without compromising training stability or convergence. Furthermore, to meet the generalization requirements of practical nickel-cobalt alloy defect detection, the proposed optimizer significantly enhances the generalization performance of YOLOv8n in this task, providing an effective training optimization solution for complex industrial defect detection. To demonstrate the feasibility of the method, the convergence proof is provided in detail. These findings highlight the value of examining generalization from the perspective of loss's time derivative and provide a new paradigm for efficiently improving the generalization capability of DNNs. The source code is available at https://github.com/LongJin-lab/NDGTA.