Evaluating Scalable Uncertainty Estimation Methods for Deep Learning-Based Molecular Property Prediction.

Area of Science:

• Computational chemistry and cheminformatics
• Machine learning and artificial intelligence
• Deep learning for scientific applications

Background:

• Deep neural networks (DNNs), particularly graph convolutional neural networks (GCNNs), achieve high accuracy in molecular property prediction.
• Quantifying prediction uncertainty remains a critical challenge, especially for out-of-domain molecules, impacting model reliability.
• Scalable methods for uncertainty estimation in DNNs are needed, with few evaluated for molecular property prediction tasks.

Purpose of the Study:

• To quantitatively compare scalable uncertainty estimation techniques for GCNNs in molecular property prediction.
• To introduce criteria for evaluating different aspects of uncertainty (aleatoric and epistemic).
• To assess the impact of uncertainty quantification on error reduction.

Main Methods:

• Theoretical comparison of MC-dropout, Deep Ensembles, and bootstrapping within a unified framework separating aleatoric and epistemic uncertainty.
• Experimental evaluation of these methods on public molecular property datasets.
• Quantitative assessment using defined criteria for uncertainty aspects and error reduction.

Main Results:

• Deep Ensembles and bootstrapping consistently demonstrated superior performance compared to MC-dropout for uncertainty estimation in GCNNs.
• The study provides a quantitative analysis of the performance of different uncertainty estimation methods and their effect on error reduction.
• Findings highlight context-specific advantages and disadvantages of each method and the challenge of out-of-domain uncertainty.

Conclusions:

• Deep Ensembles and bootstrapping are effective and scalable methods for uncertainty quantification in GCNN-based molecular property prediction.
• The research enhances understanding of aleatoric and epistemic uncertainty in relation to dataset features.
• Addressing out-of-domain uncertainty remains a significant challenge in molecular property prediction using DNNs.