Intelligent navigation of potential energy surfaces: leveraging deep reinforcement learning paradigms for accelerated...

The functional properties of nanoclusters are dictated by their atomic-scale structures; however, the efficient discovery of global energy minima on complex high-dimensional potential energy surfaces remains a formidable challenge in computational materials design. Traditional global optimization algorithms often struggled with slow convergence and a tendency to become trapped in local minima. Here, we present a deep reinforcement learning framework called the Deepcluster, which employs an agent that autonomously navigate the intricate potential energy landscapes to identify the most stable structures of nanoclusters. Our approach leverages an actor-critic network guided by the Trust Region Policy Optimization (TRPO) algorithm to intelligently balance the exploration of new configurations with the exploitation of low-energy regions. This framework combines advanced decision-making of reinforcement learning (RL) with deep learning to intelligently balance the exploration of new configurations against the exploitation of low-energy regions. Unlike supervised methods that rely on static datasets, our on-policy Deepcluster agent autonomously explores the configuration space through trial and error, trained in real-time from an initial random structure generated by the Birmingham parallel genetic algorithm. By forgoing the need for a predefined structural dataset, the agent learns to optimize configurations dynamically. This is achieved through a comprehensive state embedding that incorporates atom-centered symmetry functions (ACSFs) alongside energy, forces, and structural flags. A deep neural actor network-based on multi-layer perceptrons-then proposes optimal atomic displacements. Resulting configurations are subsequently optimized using the Effective Medium Theory (EMT) potential in conjunction with the BFGS algorithm, enabling the Deepcluster agent to efficiently navigate the energy landscape without dependence on a vast training set of pre-optimized structures. We demonstrated the power and generality of this framework by discovering the global minima (GM) structures of a series of nickel nanoclusters (Ni₁₀, Ni₁₃, Ni₂₀, Ni₃₈). The structural and thermodynamic stability of the stable strucutures identified from the Deepcluster framework is validated by first-principles calculations, including strongly negative binding energies and thermal stability from ab initio molecular dynamics simulations at 300 K. Crucially, the identified global minima show exact agreement with independent genetic algorithm searches, providing compelling cross-methodological validation. The Deepcluster agent establishes a robust, scalable, and efficient paradigm that transcends the limitations of traditional approaches, paving the way for the accelerated discovery of complex functional nanomaterials for catalysis and energy applications.