Guidelines for Recurrent Neural Network Transfer Learning-Based Molecular Generation of Focused Libraries

Area of Science:

• Computational chemistry
• Artificial intelligence in drug discovery

Background:

• Deep learning, particularly recurrent neural networks (RNNs), is increasingly used for *de novo* molecular design.
• Transfer learning, involving pre-training on large datasets and fine-tuning on smaller, specific datasets, shows promise for automated drug design.
• Assessing and comparing the performance of various deep learning methods for molecular generation remains a challenge.

Purpose of the Study:

• To investigate the impact of dataset size on the performance of transfer learning using gated recurrent unit (GRU)-RNNs for molecular generation.
• To establish guidelines for effective transfer learning in automated drug design.
• To avoid biases introduced by post-filtering in molecule generation.

Main Methods:

• Application of a GRU-RNN model for transfer learning.
• Training the model on datasets of varying sizes and complexity.
• Analysis of generated molecules and training process effectiveness.

Main Results:

• GRU-RNN-based molecular generation via transfer learning is sensitive to the size of the fine-tuning dataset.
• A minimum of 190 molecules in the fine-tuning dataset is identified as necessary for effective GRU-RNN-based molecular generation.
• Avoiding extensive post-filtering is crucial to prevent introducing biases in drug candidate selection.

Conclusions:

• Transfer learning with GRU-RNNs offers a viable approach for automated molecular generation.
• Dataset size is a critical parameter for successful transfer learning in this context.
• The findings provide practical guidelines for optimizing deep learning models in drug discovery and can be applied to benchmark other methodologies.