Computation of simple invariant solutions in fluid turbulence with the aid of deep learning

The dynamical systems view of a turbulent fluid flow provides a tantalizing connection between the self-sustaining nonlinear mechanics of turbulence and its more well-known statistical properties, and promises to open up new avenues in our ability to understand, predict and control complex fluid motion. However, successful application of these ideas to a high Reynolds number (Re) problem requires the discovery and convergence of an expansive library of simple invariant solutions (e.g. equilibria, periodic orbits). The key challenge for the field has been that algorithms to compute dynamically relevant structures struggle for a variety of reasons outside of the weakly turbulent regime. It is here that ideas from deep learning have started to show promise, and this review describes how various techniques from the machine learning community have accelerated progress. First, the use of autoencoders - neural networks which perform a nonlinear analogue to PCA - will be described. There is compelling evidence that the low-order representations of the flow learned by these models are closely connected to the unstable simple invariant solutions embedded in the turbulent attractor. As such, these representations can be used to measure shadowing of periodic solutions, to parameterize reduced order models and to estimate manifold dimension. The other key technique adapted from deep learning reviewed here is the advance in high-dimensional, gradient-based optimization that has been driven by the requirements of neural network training. To exploit these tools, the search for simple invariant solutions is converted to a hunt for minima of a scalar loss function, and gradient computation is performed efficiently within a fully differentiable flow solver. Using forced, two-dimensional turbulence as a test case, these new methods reveal an order of magnitude more solutions than has been possible using earlier approaches and converge periodic orbits where previous methods have been ineffective. An assessment will be made as to what the large set of new exact solutions says about the 'dynamical systems' exercise in general and the prospects for application at high Re.