Reliable Prediction Errors for Deep Neural Networks Using Test-Time Dropout.

Area of Science:

• Computational chemistry
• Machine learning in drug discovery
• Bioinformatics

Background:

• Deep learning shows promise in drug discovery but lacks reliable error estimation for predictions.
• Identifying unreliable predictions is crucial for applications like precision medicine.

Purpose of the Study:

• To present a novel framework for computing reliable prediction errors in neural networks.
• To enable better decision-making in drug discovery and precision medicine.

Main Methods:

• Utilized Test-Time Dropout with a single neural network, applying it multiple times (N) with dropout enabled.
• Generated an ensemble of predictions for validation and test sets.
• Employed Conformal Prediction using validation set residuals and errors to estimate test set prediction errors.

Main Results:

• Demonstrated the validity and efficiency of Dropout Conformal Predictors on 24 ChEMBL datasets.
• Showcased narrower confidence intervals compared to Random Forest-based Conformal Predictors.
• Achieved comparable compound retrieval rates in virtual screening experiments.

Conclusions:

• The proposed framework offers a computationally efficient method for generating reliable prediction errors for deep neural networks.
• This approach is broadly applicable to enhance deep neural network reliability in drug discovery and other fields.