Targeted molecular generation with latent reinforcement learning

Computational methods for generating molecules with specific physiochemical properties or biological activity can greatly assist drug discovery efforts. Deep learning generative models constitute a significant step towards that direction. We introduce a novel approach that utilizes a Reinforcement Learning paradigm, called proximal policy optimization, for optimizing molecules in the latent space of a pretrained generative model. Working in the latent space of a generative model lets us bypass the need for explicitly defining chemical rules when computationally designing molecules. The generation of molecules is achieved through navigating the latent space for identifying regions that correspond to molecules with desired properties. Proximal policy optimization is a state-of-the-art policy gradient algorithm capable of operating in continuous high-dimensional spaces in a sample-efficient manner. We have paired our optimization framework with the latent spaces of two different architectures of autoencoder models showing that the method is agnostic to the underlying architecture. We present results on commonly used benchmarks for molecule optimization that demonstrate that our method has comparable or even superior performance to state-of-the-art approaches. We additionally show how our method can generate molecules that contain a pre-specified substructure while simultaneously optimizing for molecular properties, a task highly relevant to real drug discovery scenarios.