Assessment of machine learning reliability methods for quantifying the applicability domain of QSAR regression models

Area of Science:

• Computational chemistry
• Machine learning
• cheminformatics

Background:

• Quantitative structure-activity relationship (QSAR) models are crucial for predicting molecular properties.
• Applying QSAR models requires identifying reliable domains within vast chemical space.
• Traditional QSAR reliability relies on similarity to training data, which can be limiting.

Purpose of the Study:

• To evaluate novel machine learning techniques for quantifying QSAR prediction confidence.
• To compare these methods against standard similarity-based reliability scores.
• To assess the impact of dataset characteristics and regression methods on reliability scoring.

Main Methods:

• Utilized 20 public QSAR datasets with continuous responses.
• Assessed 10 different reliability scoring methods.
• Correlated reliability scores with actual prediction errors.
• Investigated the integration of multiple reliability estimation approaches.

Main Results:

• Alternative machine learning methods for reliability scoring outperformed traditional similarity-based approaches.
• The effectiveness of reliability scoring methods is influenced by dataset properties and the chosen regression technique.
• Integrating scores from various reliability estimation methods can improve performance, despite increased computational cost.

Conclusions:

• Novel machine learning techniques offer superior reliability estimation for QSAR models compared to standard methods.
• Reliability scoring in QSAR is dataset- and model-dependent, necessitating flexible approaches.
• An open-source package for Orange data mining suite implements these advanced reliability estimation techniques.