Counting cells can accurately predict small-molecule bioactivity benchmarks

Area of Science:

• Computational chemistry
• Drug discovery
• Bioinformatics

Background:

• Accurate prediction of chemical bioactivity is crucial for efficient drug development.
• Widely used benchmark datasets for bioactivity assays often contain assays related to cell health and cytotoxicity.
• Many phenotypic assays are confounded by active compounds affecting cell count, while inactive compounds do not.

Purpose of the Study:

• To identify and mitigate biases in bioactivity assay benchmarks.
• To evaluate the added value of phenotypic profiles (mRNA, Cell Painting) beyond simple cell count.
• To propose best practices for benchmarking machine learning models in drug discovery.

Main Methods:

• Recommending filtering of benchmark datasets to exclude cell health and cytotoxicity assays.
• Implementing a cell-count baseline model for comparison.
• Utilizing a benchmark of 24 protein-target assays.
• Comparing the performance of models using Cell Painting image-based profiles against the cell-count baseline.

Main Results:

• Cell counting provides unexpectedly high performance in many existing benchmarks, masking the true predictive power of other features.
• Models leveraging Cell Painting image-based profiles significantly outperformed the cell-count baseline in a benchmark of 24 protein-target assays.
• The study highlights the need for careful benchmark curation and baseline inclusion.

Conclusions:

• Standard bioactivity assay benchmarks require filtering to remove confounding cell health and cytotoxicity assays.
• Cell Painting image-based profiles offer valuable predictive information for bioactivity beyond simple cell counts.
• Recommendations are provided for robust benchmarking of machine learning models in drug discovery to assess the utility of various data types.