Exploring Tunable Hyperparameters for Deep Neural Networks with Industrial ADME Data Sets.

Area of Science:

• Computational Chemistry
• Pharmacology
• Machine Learning

Background:

• Deep learning (DL) shows promise in drug discovery, potentially outperforming traditional machine learning on large datasets.
• Machine learning models, including deep neural networks (DNNs), are crucial for quantitative structure-activity relationship (QSAR) modeling and predicting molecular properties (ADME).
• A key challenge in building effective DNN models is identifying optimal hyperparameters for model generalization.

Purpose of the Study:

• To investigate the impact of various tunable hyperparameters on DNN model performance for QSAR modeling.
• To analyze the sensitivity and influence of specific hyperparameters on 24 industrial ADME datasets.
• To propose best practices for building robust DNN QSAR models based on empirical findings.

Main Methods:

• Systematic investigation of five key hyperparameters: learning rate, weight decay (L2 regularization), dropout rate, activation function, and batch normalization.
• Development and optimization of DNN models for 24 distinct industrial ADME datasets.
• Benchmarking optimized DNN models against established production models.

Main Results:

• Identified specific hyperparameters significantly influencing DNN model generalization for QSAR tasks.
• Demonstrated that optimized DNN models can outperform existing benchmark models in predicting ADME properties.
• Quantified the sensitivity of model performance to variations in learning rate, regularization, and network architecture choices.

Conclusions:

• Hyperparameter tuning is critical for achieving high-performance DNN-based QSAR models.
• The study provides data-driven recommendations for selecting hyperparameters to enhance model accuracy and reliability.
• These findings contribute to advancing the application of deep learning in pharmaceutical research and drug development.