Comparison of Deep Learning With Multiple Machine Learning Methods and Metrics Using Diverse Drug Discovery Data Sets

Area of Science:

• Computational chemistry and cheminformatics
• Artificial intelligence in drug discovery
• Pharmacology and toxicology

Background:

• Machine learning (ML) has been integral to pharmaceutical research for decades, particularly with fingerprint descriptors and Bayesian methods.
• Deep learning (DL), a subset of ML utilizing multi-layered neural networks, is gaining traction for drug discovery tasks like virtual screening.
• Previous work highlighted the need for comparative studies of ML methods against DL across diverse pharmaceutical datasets.

Purpose of the Study:

• To compare the performance of various machine learning methods, including deep learning, against established techniques using diverse pharmaceutical research datasets.
• To evaluate the efficacy of deep neural networks (DNNs) against methods like Support Vector Machines (SVM) for predicting endpoints such as ADME/Tox properties, antimicrobial activity, and general drug discovery data.
• To assess model performance using a comprehensive set of metrics and visualize model behavior for training and test sets.

Main Methods:

• Utilized multiple datasets covering solubility, probe-likeness, hERG, KCNQ1, and infectious diseases (bubonic plague, Chagas, tuberculosis, malaria).
• Employed FCFP6 fingerprints as molecular descriptors for all machine learning models.
• Compared Deep Neural Networks (DNNs), Support Vector Machines (SVM), and other ML algorithms, evaluating performance using metrics like AUC, F1 score, Cohen's kappa, and Matthews correlation coefficient.

Main Results:

• Deep Neural Networks (DNNs) consistently ranked higher than Support Vector Machines (SVM) and other machine learning methods based on normalized scores across various metrics and datasets.
• Radar plots effectively visualized model performance, highlighting potential issues like inferiority or over-training.
• The study demonstrated DNNs' superior performance in predicting various pharmaceutical endpoints compared to traditional ML approaches.

Conclusions:

• Deep learning, specifically DNNs, offers improved performance for diverse pharmaceutical research endpoints compared to conventional machine learning methods.
• Further extensive research is warranted to fully assess deep learning's potential, including larger-scale comparisons, prospective testing, and exploration of different DNN architectures and molecular fingerprints.
• The findings underscore the importance of using multiple performance metrics and visualization techniques for robust model evaluation in drug discovery.