Large-scale evaluation of k-fold cross-validation ensembles for uncertainty estimation

Area of Science:

• Cheminformatics
• Machine Learning
• Computational Chemistry

Background:

• Regression models often lack reliable uncertainty estimation.
• Ensemble methods are the standard for uncertainty quantification and performance enhancement.
• Subsampling-based ensembles are model-agnostic but their cheminformatics applicability is under-explored.

Purpose of the Study:

• To evaluate the applicability and effectiveness of ensemble methods in cheminformatics.
• To assess how ensemble size impacts predictive performance and uncertainty quantification.
• To provide practical recommendations for using ensembles in cheminformatics tasks.

Main Methods:

• Evaluated ensemble methods across 32 diverse cheminformatics datasets.
• Tested combinations of five modeling techniques and four molecular featurizations.
• Investigated ensemble sizes up to 200 members.

Main Results:

• Ensemble methods demonstrated broad applicability across various cheminformatics tasks.
• Increasing ensemble size generally improved predictive performance and uncertainty estimation.
• Performance gains and uncertainty quantification effectiveness varied with dataset and modeling choices.

Conclusions:

• Ensemble methods are a valuable tool for both prediction and uncertainty estimation in cheminformatics.
• The optimal ensemble size depends on whether the priority is predictive accuracy or uncertainty quantification.
• Practitioners can leverage these findings to effectively implement ensemble strategies.