A novel approach to quantify out-of-distribution uncertainty in Neural and Universal Differential Equations

Dynamical systems play a central role across the quantitative sciences, offering a powerful mathematical framework to describe, analyze, and predict the evolution of complex processes over time. In systems biology, dynamical systems provide a foundation for modeling and predicting the intricate behaviors of biological systems. Recent advances in data-driven approaches, such as Neural Ordinary Differential Equations (NODEs) and Universal Differential Equations (UDEs), have enabled the development of models that are either fully or partially data-driven. Integrating data-driven components into dynamical systems amplifies the challenge of generalization beyond training data, highlighting the need for robust methods to quantify uncertainty in out-of-distribution (OOD) scenarios-i.e., conditions not encountered during training. In this work, we investigate the reliability of uncertainty quantification (UQ) based on ensembles of models in the reconstruction of dynamical systems. We show that standard ensembles (i.e., models trained independently with different random initializations) risk producing overconfident predictions in previously unseen scenarios, as the models in the ensemble tend to exhibit similar behaviors. To address this issue, we propose a novel ensemble construction method for NODEs and UDEs that fosters diversity in the reconstructed vector field across models within specific regions of the state space, while maintaining explicit control over the fit on the training set. We first evaluate our method on numerical test cases derived from three models commonly used as benchmarks for data-driven reconstruction of dynamical systems: the Lotka-Volterra model, the damped oscillator, and the Lorenz system. We then apply the method to a biologically motivated model of cell apoptosis, considering more realistic conditions such as partial observability of the system outputs and noise in the training dataset. Overall, our results show that the proposed method improves the reliability of UQ in previously unseen scenarios compared with standard ensembles, especially where the latter exhibit overconfidence.