Enabling data-limited chemical bioactivity predictions through deep neural network transfer learning

Area of Science:

• Computational chemistry
• Cheminformatics
• Machine learning

Background:

• Developing deep neural network (DNN) models for chemical bioactivity prediction is hindered by limited assay data.
• Structural fingerprints (atom- and bond-based) are commonly used as input for feedforward DNNs.

Purpose of the Study:

• To investigate the transferability of dense layers from a pre-trained DNN to develop new DNNs for predicting other properties using limited data.
• To determine the conditions under which pre-trained DNN layers can be effectively reused for new prediction tasks.

Main Methods:

• Quantitative study involving over 400 pairs of assay datasets.
• Utilized fully trained layers from a large dataset to augment training data for smaller datasets.
• Assessed the impact of assay dataset correlation on transfer learning efficiency.

Main Results:

• Higher correlation (r) between assay datasets led to more efficient transfer learning and reduced prediction errors.
• Mean squared prediction errors decreased by 10-20% for every 0.1 increase in r^2, primarily with the first dense layer transfer.
• Deeper dense layers conveyed more specialized, assay-specific information, offering diminishing returns for transfer.

Conclusions:

• Transfer learning using pre-trained DNN layers is a viable strategy to overcome data limitations in bioactivity prediction.
• The effectiveness of transfer learning is strongly dependent on the correlation between the source and target datasets.
• Significant reductions in required training data (up to tenfold) are achievable without compromising prediction accuracy.