Analytical coarse grained potential parameterization by reinforcement learning for anisotropic cellulose

Cellulose nanocrystals (CNCs) are a type of cellulose with excellent mechanical performance and other advantageous attributes. According to previous reports, hydrogen bonds play a pivotal role in the anisotropic structure of the CNC. Understanding the structure and mechanical behavior of CNC on a mesoscopic scale is critical for the development and manufacture of cellulose materials. However, experimental observations and atomistic simulations are not appropriate on the mesoscopic scale. In this study, we introduce an analytical coarse-grained (CG) potential following an extended bottom-up approach that is directly parameterized using reinforcement learning (RL). RL is a powerful tool for industrial and academic applications in various fields. Nevertheless, the potential of RL has not yet been fully exploited in the field of molecular dynamics. The RL and Boltzmann inversion methods were employed to develop a novel CG model of cellulose to represent its anisotropy and polymer stiffness. The resulting approximate CG model is not limited to the specific properties used for training; it can reproduce the dynamic mechanical properties under various conditions without further optimization. This model confirms that RL can construct a CG potential that is both physically interpretable and powerful. The training code is available at https://github.com/EiPiFun/rl-cll-cg .

All atom (AA) reference molecular dynamics simulations were conducted using GROMACS with CHARMM36 force field, utilizing CHARMM-GUI tools to generate the necessary force field files. Coarse-grained (CG) simulations were performed using LAMMPS, employing a custom-defined explicit analytical force field. This potential consists of harmonic bonded interactions, 12-6 Lennard-Jones nonbonded interactions, and a modified 12-10 potential to account for directional hydrogen bonding. Initial estimates for bonded force constants were derived using the Boltzmann inversion method. The CG potential parameters were parameterized via a degenerate reinforcement learning (RL) approach employing the soft actor-critic (SAC) algorithm, implemented through the OpenAI Stable-Baselines3. To assess mechanical performance and model generalization, steered molecular dynamics techniques were utilized to simulate in-plane and out-of-plane fractures. For benchmarking purposes, baseline CG models were generated using the iterative Boltzmann inversion (IBI), relative entropy (RE), and force matching (FM) approaches via VOTCA, while the MARTINI 3 model was prepared using Polyply. Further validation of the optimization strategy was performed by comparing the RL approach against Tree-structured Parzen Estimator (TPE) and Covariance Matrix Adaptation Evolution Strategy (CMAES) algorithms using OPTUNA. All molecular structures were visualized using Open-Source PyMOL.