Reinforcement learning for in silico determination of adsorbate-substrate structures

Area of Science:

• Computational chemistry
• Materials science
• Artificial intelligence

Background:

• Reinforcement learning (RL) has achieved state-of-the-art results in artificial intelligence, notably after breakthroughs like AlphaGo.
• Determining the stable structures of adsorbates on substrates is crucial for understanding surface interactions and designing new materials.
• Traditional methods for structural determination can be computationally expensive and time-consuming.

Purpose of the Study:

• To present a novel RL-based method for the in silico structural determination of adsorbate@substrate models.
• To develop and implement this RL method within the RLMaterial software for materials design.
• To demonstrate the applicability of the method across diverse chemical systems.

Main Methods:

• A Q-learning based reinforcement learning approach was employed to navigate the energy landscape of adsorbate@substrate interactions.
• The RL agent performs actions (translations, rotations) to minimize energy, guided by a learned policy.
• RLMaterial software interfaces with computational chemistry codes (deMon2k, DFTB+, ORCA, Quantum Espresso) for energy calculations.
• Artificial neural networks and gradient boosting regression were used to approximate the Q-matrix for efficient decision-making.
• Density Functional Tight Binding (DFTB) and Density Functional Theory (DFT) were utilized for energy computations.

Main Results:

• The RL method successfully determined the structures of glycine and 2-amino-acetaldehyde on a boron nitride monolayer.
• It also elucidated host-guest interactions between phenylboronic acid and β-cyclodextrin, and ammonia on naphthalene.
• The use of machine learning techniques (ANN, gradient boosting) improved the efficiency of the Q-matrix approximation.
• A transfer-learning protocol was successfully developed, enabling knowledge transfer between different chemical systems and computational levels.

Conclusions:

• The developed RL method offers an efficient and effective approach for the structural determination of adsorbate@substrate systems.
• RLMaterial provides a versatile platform for accelerating materials discovery through AI-driven simulations.
• The transfer-learning capability enhances the adaptability and predictive power of RL in computational chemistry.