Reinforcement Learning Based Adaptive Sampling: REAPing Rewards by Exploring Protein Conformational Landscapes.

Area of Science:

• Computational chemistry
• Biophysics
• Machine learning applications in science

Background:

• Molecular Dynamics (MD) simulations face computational challenges in sampling protein conformational landscapes, especially for large systems or long timescales.
• Efficiently exploring these landscapes is crucial for understanding protein function and dynamics.

Purpose of the Study:

• To introduce the REinforcement learning based Adaptive samPling (REAP) algorithm for overcoming MD sampling limitations.
• To demonstrate REAP's ability to efficiently sample conformational space by learning the importance of order parameters.

Main Methods:

• The REAP algorithm utilizes reinforcement learning principles to guide sampling.
• It rewards exploration of significant degrees of freedom and downweights less informative ones.
• Effectiveness was validated against continuous MD and least-counts adaptive sampling on model and realistic systems.

Main Results:

• REAP consistently explored conformational space faster than continuous MD and least-counts adaptive sampling across all tested systems.
• The algorithm demonstrated superior performance in discovering landscape features within a given simulation time.
• On-the-fly estimation of collective variable importance was a key advantage.

Conclusions:

• REAP offers a significant improvement in the efficiency of conformational landscape sampling for MD simulations.
• Its adaptive nature makes it particularly valuable for systems with limited prior structural information.
• The algorithm provides a powerful tool for accelerating molecular simulations in biophysics and computational chemistry.