Comparing massively-multitask regression algorithms for drug discovery

Area of Science:

• Computational Chemistry
• Cheminformatics
• Machine Learning in Drug Discovery

Background:

• Massively-multitask regression models (MMRMs) have emerged as powerful tools for predicting compound bioactivity in drug discovery.
• These models, trained on extensive datasets, offer accuracy comparable to experimental measurements.

Purpose of the Study:

• To compare the performance of six leading MMRMs (pQSAR, Alchemite, MT-DNN, MetaNN, Macau, IMC) for bioactivity profile imputation.
• To evaluate the impact of different training/test set splits on MMRM performance and accuracy estimation.

Main Methods:

• Six MMRMs were trained by experts on identical datasets comprising 159 kinase and 4276 ChEMBL assays.
• Models were evaluated using both 75/25 and 99+/ <1% training/test set splits to assess performance under varying data availability scenarios.
• Comparative analysis included qualitative assessment and statistical rigor, benchmarking against single-task random forest regression (ST-RFR).

Main Results:

• MMRMs significantly outperformed the ST-RFR model in bioactivity profile imputation.
• Performance varied considerably based on the training/test split; 75/25 splits led to a substantial underestimation of model accuracy compared to 99+/ <1% splits.
• While MMRMs excel at imputing profiles within the training data distribution, their advantage diminishes for compounds dissimilar to the training set.

Conclusions:

• MMRMs are highly effective for tasks like hit-finding, off-target prediction, and drug repurposing within the chemical space of the training data.
• The choice of data splitting strategy critically impacts the perceived accuracy of MMRMs, necessitating the use of realistic, smaller test sets for reliable evaluation.
• MMRMs' utility is greatest for exploring known chemical spaces, while their performance on novel chemical entities requires further investigation.