Achieving well-informed decision-making in drug discovery: a comprehensive calibration study using neural...

Area of Science:

• Computational chemistry
• Machine learning in drug discovery
• Biopharmaceutical development

Background:

• Drug discovery relies on predictive computational models, but their uncertainty estimates are often unreliable due to poor calibration.
• Accurate uncertainty quantification is crucial for risk assessment and decision-making in therapeutic agent development.

Purpose of the Study:

• To compare model selection strategies for achieving well-calibrated neural networks.
• To introduce and evaluate the HMC Bayesian Last Layer (HBLL) method for efficient Bayesian uncertainty estimation.
• To assess the impact of combining post hoc calibration with uncertainty quantification methods.

Main Methods:

• Comparative analysis of hyperparameter tuning metrics (accuracy, calibration scores).
• Implementation of the HMC Bayesian Last Layer (HBLL) method using Hamiltonian Monte Carlo (HMC) sampling.
• Integration of post hoc calibration techniques with uncertainty quantification approaches.

Main Results:

• The HBLL method significantly improves model calibration compared to baseline neural networks.
• HBLL achieves performance comparable to existing uncertainty quantification methods.
• Combining post hoc calibration with uncertainty quantification boosts both model accuracy and calibration.

Conclusions:

• The HBLL method offers an efficient approach to enhance uncertainty estimation in predictive models for drug discovery.
• Well-calibrated models are essential for reliable risk assessment in therapeutic development.
• Hybrid approaches combining calibration and uncertainty quantification yield superior model performance.