Parsimonious Optimization of Multitask Neural Network Hyperparameters.

Area of Science:

• Computational Chemistry
• Machine Learning
• Cheminformatics

Background:

• Neural networks are increasingly used in chemical modeling and Quantitative Structure-Activity Relationship (QSAR) studies.
• Hyperparameter tuning is essential for neural network performance but computationally expensive.

Purpose of the Study:

• Compare four hyperparameter optimization methods: grid search, random search, tree-structured Parzen estimator, and genetic algorithms.
• Evaluate optimization efficiency in terms of computational time and model performance.
• Investigate hyperparameter influence using experimental design.

Main Methods:

• Applied grid search, random search, tree-structured Parzen estimator, and genetic algorithms.
• Utilized three multitask QSAR datasets for evaluation.
• Employed experimental design strategies to analyze hyperparameter effects.

Main Results:

• Genetic algorithms, tree-structured Parzen estimator, and random search reduced computational time by 99.92% compared to grid search.
• Tree-structured Parzen estimator and genetic algorithms outperformed random search in performance.
• Experimental design identified key hyperparameter influences.

Conclusions:

• Advanced optimization techniques significantly reduce computational cost for neural network hyperparameter tuning.
• Tree-structured Parzen estimator and genetic algorithms present efficient and effective strategies for multitask QSAR model optimization.