Related Experiment Video
Updated: Jun 30, 2026
09:01
A High-throughput Assay for the Prediction of Chemical Toxicity by Automated Phenotypic Profiling of Caenorhabditis elegans
Published on: March 14, 2019
Tox21mer, A transformer foundation model for Tox21 high-throughput concentration-response curves data
Leping Li1, Jisoo Hwang1, Keith Shockley1
1Biostatistics and Computational Biology Branch.
Biorxiv : the Preprint Server for Biology
|June 29, 2026
Summary
Tox21mer, a new transformer model, effectively encodes chemical assay data, enabling accurate prediction of compound activity and potency. This self-supervised approach provides a robust foundation for analyzing high-throughput screening results.
Area of Science:
- Computational toxicology
- Cheminformatics
- Machine learning in drug discovery
Background:
- The U.S. Tox21 collaboration has amassed a substantial library of high-throughput screening (HTS) concentration-response assay data.
- Developing effective computational models to interpret and leverage this complex data is crucial for advancing chemical safety and drug discovery.
Purpose of the Study:
- To introduce Tox21mer, a novel transformer model designed to encode Tox21 concentration-response curves and associated metadata.
- To evaluate the performance of Tox21mer's learned representations in predicting assay outcomes and compound potency.
Main Methods:
- Tox21mer, a 43.5-million-parameter transformer, was pre-trained on approximately 2.5 million concentration-response curves using a masked-response reconstruction objective.
- Auxiliary supervision was applied to assay outcome and AC50 values during pre-training.
- Learned embeddings were evaluated by training lightweight probes on held-out compound data.
Main Results:
- The learned representation achieved high performance in predicting three-class outcomes (macro-F1: 0.985), binary active/inactive status (binary F1: 0.994), and log10(AC50) (R^2: 0.87).
- Embeddings exhibited coherent clustering based on curve-class categories.
- Ablation studies indicated that performance relies primarily on response-value distributions within the assay context.
Conclusions:
- Tox21mer provides a powerful, reusable foundation representation for Tox21 concentration-response data.
- This model facilitates extrapolation to untested compounds and can be integrated into downstream applications for large-scale screening.
