Deep latent force models: ODE-based process convolutions for Bayesian deep learning.

Area of Science:

• Machine Learning
• Dynamical Systems Modeling
• Uncertainty Quantification

Background:

• Modeling highly nonlinear dynamical systems with robust uncertainty quantification is complex.
• Existing methods often require problem-specific designs.
• A domain-agnostic approach is needed for broader applicability.

Purpose of the Study:

• Introduce a domain-agnostic model for nonlinear dynamical systems.
• Develop a deep Gaussian process with physics-informed kernels.
• Enable robust uncertainty quantification in complex system modeling.

Main Methods:

• Developed the deep latent force model (DLFM), a deep Gaussian process.
• Incorporated physics-informed kernels derived from ordinary differential equations.
• Utilized two formulations: weight-space and variational inducing points.
• Employed doubly stochastic variational inference for model approximation.

Main Results:

• DLFM effectively captures dynamics in highly nonlinear, multi-output time-series data.
• Achieved comparable performance to non-physics-informed models on regression tasks.
• Identified a negative impact of inducing points on extrapolation capabilities.

Conclusions:

• The DLFM offers a powerful, domain-agnostic solution for modeling complex dynamical systems.
• Physics-informed kernels enhance the model's ability to capture system dynamics and quantify uncertainty.
• Further research is needed to optimize extrapolation performance.