A meta-learning framework to mitigate negative transfer in transfer learning applicable to drug design

Data sparseness is a major limiting factor for deep machine learning. In the natural sciences, data distributions are heterogeneous. For instance, in chemistry and early-phase drug discovery, compound and molecular property data are typically sparse compared to data in other fields such as particle physics or genome biology. For machine learning in low-data regimes, approaches such as transfer learning or meta-learning have been introduced. These learning strategies are conceptually related but algorithmically distinct and typically applied independently. They share the common goal of facilitating knowledge transfer between domains with related prediction tasks and varying data availability. We were interested in combining meta- and transfer learning into a coherent framework, primarily for deep learning in cheminformatics. Therefore, we introduce a new meta-learning algorithm designed to complement transfer learning. It identifies an optimal subset of training instances and determines weight initializations for deriving base models that can then be fine-tuned under conditions of data scarcity. Given its ability to identify preferred training samples, the meta-learning algorithm balances negative transfer between source and target domains, which represents a major caveat for transfer learning. In an extensive proof-of-concept application, inhibitors of protein kinases were predicted following data reduction using combined meta- and transfer learning, revealing statistically significant increases in model performance and effective control of negative transfer.