Stochasticity and probabilistic trajectory scoring are essential for data-driven closures of chaotic systems

Coarse-grained models of chaotic systems neglect unresolved degrees of freedom, inducing structured model error that limits predictability and distorts long-term statistics. Typical data-driven closures are trained to minimize prediction error over a single time step, implicitly assuming Markovian dynamics and often failing to capture long-term behavior. Recent approaches instead optimize losses over finite trajectories. However, when such trajectory-based training is carried out with deterministic pointwise losses, it introduces a fundamental mathematical degeneracy. We prove that optimizing pointwise deterministic losses, including but not limited to mean squared error, over chaotic trajectories suppresses predictive variance, with a corresponding loss of physical variability in long integrations. In contrast, strictly proper scoring rules avoid this degeneracy. By targeting forecast distributions rather than realized trajectories, they remove the penalty against predictive spread and align the long-lead optimum with the invariant measure. Using quasi-geostrophic turbulence as a canonical chaotic system, we validate this theory: Closures trained with one-step losses fail to capture stable coarse-grained dynamics, while deterministic closures optimized over trajectories exhibit the variance-loss tendency predicted by our analysis. Stochastic closures calibrated over trajectories using the energy score, however, overcome both structural limitations, yielding skillful ensemble forecasts and realistic long-term statistics. Our results establish that both stochastic modeling and trajectory-based calibration are essential for faithfully representing the dynamics of coarse-grained systems.