Model fitting for small skin permeability data sets: hyperparameter optimisation in Gaussian Process Regression

Area of Science:

• Computational chemistry
• Machine learning
• Statistical modeling

Background:

• Gaussian Process (GP) models are powerful tools for predictive modeling.
• Optimizing hyperparameters is crucial for GP model performance.
• Quantitative Structure-Permeability Relationships (QSPR) are vital in drug discovery.

Purpose of the Study:

• To enhance predictions from Gaussian Process models through hyperparameter optimization.
• To evaluate various optimization techniques for GP models.
• To assess the impact of dataset size and chemical space on model quality.

Main Methods:

• Compared Grid Search, Conjugate Gradient, Random Search, Evolutionary Algorithm, and Hyper-prior optimization methods.
• Applied methods to previously published datasets.
• Systematically reduced dataset sizes while preserving descriptor range ('chemical space').

Main Results:

• The Hyper-prior Smoothbox kernel yielded the best performing models across most datasets.
• GP models with optimized hyperparameters significantly outperformed benchmark QSPR models.
• Optimization methods generally maintained statistical quality even with reduced dataset sizes.

Conclusions:

• Dataset design and validation approach are critical for developing improved predictive models.
• Dataset size is not always a limiting factor for model quality if data is well-controlled.
• High-quality models can be developed from substantially smaller datasets.