Selecting Relevant Descriptors for Classification by Bayesian Estimates: A Comparison with Decision Trees and Support...

Area of Science:

• Computational chemistry
• Machine learning in drug discovery

Background:

• Classification algorithms face overfitting due to high dimensionality.
• Identifying relevant descriptors is crucial for reducing complexity and improving model performance.

Purpose of the Study:

• To apply Bayesian estimates for modeling descriptor probability distributions in binary classification.
• To assess the discriminative power of descriptors using Kullback-Leibler divergence.
• To compare Bayesian estimates with LASSO, decision trees, and support vector machines on unbalanced datasets.

Main Methods:

• Utilized Bayesian estimates to model descriptor value distributions for binary classification.
• Employed n-fold cross-validation to evaluate classifier performance.
• Calculated symmetric Kullback-Leibler divergence to measure discriminative power.
• Compared results with LASSO feature selection, decision trees, and support vector machines.

Main Results:

• Relevant descriptors exhibit Gaussian-like distributions and large Kullback-Leibler divergences.
• Bayesian estimates demonstrate robustness against unbalanced data, unlike decision trees and support vector machines.
• Identified descriptors enabling linear class separation.

Conclusions:

• Bayesian estimates provide a robust method for feature selection in classification tasks, particularly with imbalanced data.
• The approach effectively identifies descriptors that facilitate linear separation, reducing the risk of overfitting.
• This strategy aids in understanding the underlying chemical space relevant for classification.