Augmented Memory: Sample-Efficient Generative Molecular Design with Reinforcement Learning.

Area of Science:

• Computational chemistry
• Artificial intelligence in drug discovery
• Materials informatics

Background:

• Sample efficiency is a critical challenge in *de novo* molecular design, especially when using computationally expensive oracles.
• Existing molecular generative models, particularly those using simplified molecular-input line-entry system (SMILES) with reinforcement learning, show promise but can be further optimized.
• High-accuracy oracles often require significant computational resources, limiting practical molecular optimization within a feasible budget.

Purpose of the Study:

• To enhance the sample efficiency of molecular generative models.
• To develop a novel algorithm that minimizes calls to expensive computational property predictors (oracles).
• To establish a new state-of-the-art in *de novo* molecular design.

Main Methods:

• Implemented experience replay to improve existing algorithms.
• Proposed a novel algorithm, Augmented Memory, combining data augmentation with experience replay.
• Demonstrated the reusability of oracle scores for multiple model updates.

Main Results:

• Experience replay significantly boosted the performance of several prior algorithms.
• Augmented Memory demonstrated substantially enhanced sample efficiency across various tasks.
• Achieved state-of-the-art performance in sample-efficient *de novo* molecular design.

Conclusions:

• Augmented Memory offers a significant advancement in sample-efficient molecular design.
• The algorithm is effective for exploitation tasks, drug discovery, and materials design.
• This approach enables optimization with costly oracles under practical computational constraints.