Rare-Event Sampling using a Reinforcement Learning-Based Weighted Ensemble Method.

Area of Science:

• Computational Chemistry
• Biophysics
• Machine Learning

Background:

• Path sampling methods are powerful for simulating rare events but are limited by the difficulty of identifying effective progress coordinates.
• Identifying a suitable progress coordinate that accurately captures the slow, relevant motions of a rare event remains a significant challenge.

Purpose of the Study:

• To develop a weighted ensemble (WE) path sampling strategy that leverages reinforcement learning (RL) to automatically identify optimal progress coordinates during simulations.
• To demonstrate the efficacy of this RL-enhanced WE strategy in accurately simulating rare events across various systems.

Main Methods:

• Developed a weighted ensemble (WE) path sampling strategy incorporating reinforcement learning (RL) for automated progress coordinate identification.
• Applied the RL-WE strategy to three benchmark systems: an egg carton potential, an S-shaped potential, and a dimer of the HIV-1 capsid protein.
• Utilized discrete-state synthetic molecular dynamics trajectories generated from a fine-grained Markov state model for efficient atomic-level simulations of the HIV-1 capsid.

Main Results:

• The RL-WE strategy successfully and automatically identified relevant progress coordinates from a set of candidates during simulations.
• The method demonstrated effectiveness across diverse systems, including complex biomolecular simulations.
• Rigorous weighting of trajectories ensured the maintenance of accurate kinetics throughout the simulations.

Conclusions:

• Reinforcement learning integrated with weighted ensemble path sampling offers an automated and effective approach to identifying progress coordinates for rare event simulations.
• This strategy overcomes a key limitation in path sampling, improving the simulation of complex molecular dynamics.
• The approach maintains simulation rigor and kinetic accuracy, paving the way for more powerful rare event simulations.