在连续深度模型中增强计算复杂性:神经常规微分方程与可训练的数值方案.
IEEE transactions on pattern analysis and machine intelligence
|August 15, 2025
概括
我们为神经普通微分方程 (NODE) 引入可训练的数值集成方案,提高计算效率. 这种新的方法通过固定函数评估实现了最先进的准确性,提高了NODE的性能.
科学领域:
- 机器学习 机器学习
- 数字分析 数字分析
- 动态系统 动态系统
背景情况:
- 神经常规微分方程 (NODE) 是连续时间的神经网络,提供系统理论见解.
- 节点对时间序列,预测和可逆网络应用非常有价值.
- 目前的NODE培训和推理由于自适应式步骤大小解答器和高数量的函数评估 (NFE) 而是计算密集的.
研究的目的:
- 开发一种新的方法来提高NODE的计算效率.
- 为了解决NODE模型中传统的自适应阶段大小解决器的性能限制.
- 通过更快,更高效的计算,提高NODE的实际应用性.
主要方法:
- 建议使数字集成方案的参数可以在NODE中进行训练.
- 开发可训练的固定步骤大小解决器,可以动态地适应NODE动态.
- 将拟议的可训练解决方案与最先进的方法比较,例如Dormand-Prince 5 ((4) (DOPRI).
主要成果:
- 在各种基准上实现了最先进的性能,包括分类,密度估计和动态系统建模.
- 通过使用固定的NFE来操作,证明了更高的计算效率.
- 展示了可训练的解决方案适应NODE动态,改善整体性能.
结论:
- 可训练的数值集成方案为高效的NODE建模提供了一个有希望的方向.
- 提出的方法显著提高了计算效率,同时保持了高精度.
- 这项工作为NODE的更实用和更广泛的应用铺平了道路.
相关概念视频
Mechanistic Models: Compartment Models in Algorithms for Numerical Problem Solving
101
Mechanistic models play a crucial role in algorithms for numerical problem-solving, particularly in nonlinear mixed effects modeling (NMEM). These models aim to minimize specific objective functions by evaluating various parameter estimates, leading to the development of systematic algorithms. In some cases, linearization techniques approximate the model using linear equations.
In individual population analyses, different algorithms are employed, such as Cauchy's method, which uses a...
In individual population analyses, different algorithms are employed, such as Cauchy's method, which uses a...
101
Linear Approximation in Frequency Domain
131
Linear systems are characterized by two main properties: superposition and homogeneity. Superposition allows the response to multiple inputs to be the sum of the responses to each individual input. Homogeneity ensures that scaling an input by a scalar results in the response being scaled by the same scalar.
In contrast, nonlinear systems do not inherently possess these properties. However, for small deviations around an operating point, a nonlinear system can often be approximated as linear....
In contrast, nonlinear systems do not inherently possess these properties. However, for small deviations around an operating point, a nonlinear system can often be approximated as linear....
131
Transmission-Line Differential Equations
403
Transmission lines are essential components of electrical power systems. They are characterized by the distributed nature of resistance (R), inductance (L), and capacitance (C) per unit length. To analyze these lines, differential equations are employed to model the variations in voltage and current along the line.
Line Section Model
A circuit representing a line section of length Δx helps in understanding the transmission line parameters. The voltage V(x) and current i(x) are measured...
Line Section Model
A circuit representing a line section of length Δx helps in understanding the transmission line parameters. The voltage V(x) and current i(x) are measured...
403
Linear Approximation in Time Domain
125
Nonlinear systems often require sophisticated approaches for accurate modeling and analysis, with state-space representation being particularly effective. This method is especially useful for systems where variables and parameters vary with time or operating conditions, such as in a simple pendulum or a translational mechanical system with nonlinear springs.
For a simple pendulum with a mass evenly distributed along its length and the center of mass located at half the pendulum's length,...
For a simple pendulum with a mass evenly distributed along its length and the center of mass located at half the pendulum's length,...
125
Differential Form of Maxwell's Equations
640
James Clerk Maxwell (1831–1879) was one of the significant contributors to physics in the nineteenth century. He is probably best known for having combined existing knowledge of the laws of electricity and the laws of magnetism with his insights to form a complete overarching electromagnetic theory, represented by Maxwell's equations. The four basic laws of electricity and magnetism were discovered experimentally through the work of physicists such as Oersted, Coulomb, Gauss, and...
640
Difference Equation Solution using z-Transform
368
The z-transform is a powerful tool for analyzing practical discrete-time systems, often represented by linear difference equations. Solving a higher-order difference equation requires knowledge of the input signal and the initial conditions up to one term less than the order of the equation.
The z-transform facilitates handling delayed signals by shifting the signal in the z-domain, which corresponds to delaying the signal in the time domain, and advancing signals by similarly shifting in the...
The z-transform facilitates handling delayed signals by shifting the signal in the z-domain, which corresponds to delaying the signal in the time domain, and advancing signals by similarly shifting in the...
368


