Sample efficient reinforcement learning with active learning for molecular design.

Area of Science:

• Computational Chemistry and Drug Discovery
• Artificial Intelligence in Scientific Research
• Machine Learning for Molecular Design

Background:

• Reinforcement learning (RL) is effective for high-dimensional problems but struggles with sample efficiency in complex scientific environments.
• Drug discovery requires efficient exploration of vast chemical spaces and multi-parameter optimization (MPO).
• Current in silico methods like virtual screening and de novo generation need enhanced sample efficiency for complex models.

Purpose of the Study:

• To improve the sample efficiency of reinforcement learning (RL) for molecular design.
• To develop and evaluate a novel active learning (AL) system integrated with RL (RL-AL) for multi-parameter optimization (MPO).
• To address challenges in combining RL and AL for efficient molecular discovery.

Main Methods:

• Integration of an active learning (AL) system with a reinforcement learning (RL) model, termed RL-AL.
• Development of a novel AL approach specifically designed to tackle the multi-parameter optimization (MPO) problem in molecular design.
• Comparative analysis against baseline RL using ligand- and structure-based oracle functions.

Main Results:

• RL-AL demonstrated a 5-66 fold increase in generated hits for a fixed oracle budget and a 4-64 fold reduction in computational time.
• Compounds discovered via RL-AL showed significant enrichment of a multi-parameter scoring objective, indicating superior performance in identifying high-scoring molecules.
• The RL-AL approach maintained output diversity while enhancing the efficacy of high-scoring compound curation.

Conclusions:

• The RL-AL system substantially accelerates the search for novel molecular solutions compared to baseline RL.
• This approach enhances the feasibility of computationally expensive oracle functions, previously limited by high costs.
• The RL-AL methodology is broadly applicable to any RL domain requiring improved sample efficiency and optimization.