Best practices for resilient automation of biomolecular simulations

As molecular dynamics simulation projects require ever-increasing amounts of simulation time, they become more complex to implement and more susceptible to unintended interruptions. This makes the task of the investigator in managing such projects more difficult and time-consuming. Here, we present a few potential operational problems that commonly arise in such simulation projects, possibly unanticipated considerations for solutions to these problems, and discussions of potential solutions that provide fault tolerance and automation. Example scripts are provided, along with the rationale for their design.