Benchmarking methods and data sets for ligand enrichment assessment in virtual screening

Area of Science:

• Computational chemistry
• Drug discovery
• Bioinformatics

Background:

• Retrospective virtual screening (VS) using benchmarking datasets is common for evaluating VS approaches.
• Existing benchmarking sets may not accurately reflect real-world screening libraries, leading to biased assessments.
• Key biases include analogue bias, artificial enrichment, and false negatives.

Purpose of the Study:

• To critically review the history and limitations of current VS benchmarking methods and datasets.
• To introduce a novel algorithm for constructing maximum-unbiased benchmarking sets for VS.
• To apply and validate this algorithm for structure-based and ligand-based VS targeting HDAC1, HDAC6, and HDAC8.

Main Methods:

• Comprehensive review of VS benchmarking history, datasets, and identified biases.
• Development of a new algorithm to generate maximum-unbiased benchmarking sets.
• Application of the algorithm to human histone deacetylase (HDAC) isoforms (HDAC1, HDAC6, HDAC8) and validation using leave-one-out cross-validation (LOO CV).

Main Results:

• Identified and categorized three primary biases in conventional VS benchmarking sets: analogue bias, artificial enrichment, and false negatives.
• Demonstrated the effectiveness of the developed algorithm in creating maximum-unbiased benchmarking sets.
• LOO CV confirmed the unbiased nature of the generated sets through property matching, ROC curves, and AUC values for HDAC targets.

Conclusions:

• Conventional VS benchmarking sets can introduce significant biases, potentially misrepresenting the true performance of VS methods.
• The newly developed algorithm effectively generates maximum-unbiased benchmarking sets suitable for both ligand-based and structure-based VS.
• This approach enhances the reliability of VS performance evaluation, particularly for drug discovery targets like HDACs.