Augmented Hill-Climb increases reinforcement learning efficiency for language-based de novo molecule generation

Area of Science:

• Artificial Intelligence
• Drug Discovery
• Computational Chemistry

Background:

• AI-driven de novo molecule generation is crucial for drug design.
• Reinforcement learning (RL) is a popular approach but can be sample-inefficient.
• Expensive scoring functions limit the scalability of current RL methods.

Purpose of the Study:

• To introduce Augmented Hill-Climb, a novel RL strategy for efficient de novo molecule generation.
• To improve sample-efficiency and optimization capabilities compared to existing methods.
• To enhance the accessibility of computationally intensive scoring functions in drug discovery.

Main Methods:

• Developed Augmented Hill-Climb, a hybrid RL strategy combining REINVENT and Hill-Climb.
• Benchmarked Augmented Hill-Climb against REINFORCE, REINVENT (v1 & v2), Hill-Climb, and best agent reminder.
• Evaluated performance on various docking tasks using recurrent neural networks and transformer architectures.
• Tuned diversity filters to address potential failure modes.

Main Results:

• Augmented Hill-Climb demonstrated a 1.5-fold improvement in optimization ability and a 45-fold improvement in sample-efficiency over REINVENT.
• Outperformed other RL strategies on six tasks, particularly in early training stages and for complex objectives.
• Achieved improved performance on both recurrent neural networks and transformer architectures.
• Generated high-quality molecules with appealing chemical properties.

Conclusions:

• Augmented Hill-Climb significantly enhances sample-efficiency for language-based de novo molecule generation via RL.
• The method makes computationally expensive scoring functions, such as docking, more feasible within practical timescales.
• This advancement offers a more efficient and accessible approach to AI-driven drug discovery.