Training a neural network to predict dynamics it has never seen

Neural networks have proven to be remarkably successful for a wide range of complicated tasks, from image recognition and object detection to speech recognition and machine translation. One of their successes lies in their ability to predict future dynamics given a suitable training data set. Previous studies have shown how echo state networks (ESNs), a type of recurrent neural networks, can successfully predict both short-term and long-term dynamics of even chaotic systems. This study shows that, remarkably, ESNs can successfully predict dynamical behavior that is qualitatively different from any behavior contained in their training set. Evidence is provided for a fluid dynamics problem where the flow can transition between laminar (ordered) and turbulent (seemingly disordered) regimes. Despite being trained on the turbulent regime only, ESNs are found to predict the existence of laminar behavior. Moreover, the statistics of turbulent-to-laminar and laminar-to-turbulent transitions are also predicted successfully. The utility of ESNs in acting as early-warning generators for transition is discussed. These results are expected to be widely applicable to data-driven modeling of temporal behavior in a range of physical, climate, biological, ecological, and finance models characterized by the presence of tipping points and sudden transitions between several competing states.