Benchmarking compound activity prediction for real-world drug discovery applications

Area of Science:

• Computational chemistry
• Drug discovery
• Bioinformatics

Background:

• Accurate prediction of compound activity is crucial for early drug discovery.
• Data-driven computational methods show promise but lack robust evaluation benchmarks.
• Existing benchmarks may not reflect real-world data complexities, leading to performance overestimation.

Purpose of the Study:

• To introduce the Compound Activity benchmark for Real-world Applications (CARA).
• To provide a comprehensive evaluation framework for computational compound activity prediction models.
• To account for biases in real-world activity data and improve model assessment.

Main Methods:

• Developed CARA by distinguishing assay types and designing specific train-test splits.
• Selected appropriate evaluation metrics to handle biased data distributions.
• Evaluated current prediction models and few-shot learning strategies on the CARA benchmark.

Main Results:

• Current compound activity prediction models show variable performance across different assays.
• The CARA benchmark helps identify limitations and overestimations in model performance.
• Few-shot training strategies exhibit task-dependent performance variations.

Conclusions:

• CARA offers a high-quality dataset for developing and validating compound activity prediction models.
• The benchmark facilitates a more realistic evaluation of computational methods in drug discovery.
• Findings suggest improvements for applying data-driven models to real-world drug discovery challenges.