Stable and efficient learning of degenerate stochastic differential equation in neural networks

Stochastic differential equations are fundamental for modeling complex dynamic systems subject to random noise. However, learning stochastic dynamics from empirical data remains challenging, particularly under degenerate noise conditions where the diffusion matrix is irreversible. Traditional approaches often suffer from numerical instability and prohibitive computational costs in such scenarios. To address these limitations, we propose three novel algorithms tailored for efficient and stable learning of SDEs with degenerate diffusion. Algorithm I integrates numerical techniques from second-order stochastic differential equation (SDE) solvers into a specialized learning framework, targeting the special case of a two-dimensional degenerate stochastic equation. This approach improves computational efficiency by taking advantage of the properties of the physical system. Algorithm II introduces a novel loss function that combines mean squared error and maximum likelihood estimation, specifically designed for systems with degenerate diffusion matrices, improving robustness by explicitly accounting for noise structure in the optimization process. Algorithm III incorporates a stabilizing auxiliary noise mechanism during training, which mitigates gradient instability without sacrificing convergence guarantees. Extensive numerical experiments demonstrate that the proposed methods significantly outperform conventional maximum likelihood estimation in terms of training stability, faster convergence, and higher accuracy. These advances enable reliable modeling of high-dimensional stochastic systems and offer promising tools for neural network learning tasks requiring robust noise adaptation.