Related Experiment Video
Updated: Nov 21, 2025
Quantitative Structure-Activity Relationship, Activity Prediction, and Molecular Dynamics of Non-nucleotide Reverse Transcriptase Inhibitors
Published on: May 9, 2025
Transformer-CNN: Swiss knife for QSAR modeling and interpretation
Pavel Karpov1,2, Guillaume Godin3, Igor V Tetko4,5
1Institute of Structural Biology, Helmholtz Zentrum München-Research Center for Environmental Health (GmbH), Ingolstädter Landstraße 1, 85764, Neuherberg, Germany. pavel.karpov@helmholtz-muenchen.de.
We developed a Transformer-CNN method using SMILES-embeddings for quantitative structure-activity relationship (QSAR) and quantitative structure-property relationship (QSPR) modeling. This approach yields interpretable models, even for small datasets, by leveraging transfer learning and data augmentation.
Area of Science:
- Computational chemistry
- Cheminformatics
- Machine learning in drug discovery
Background:
- Quantitative structure-activity relationship (QSAR) and quantitative structure-property relationship (QSPR) models are crucial for predicting molecular properties.
- Interpretable models are highly desirable for understanding structure-activity relationships.
- Existing methods may struggle with small datasets or lack interpretability.
Purpose of the Study:
- To develop a novel, interpretable QSAR/QSPR modeling approach using Transformer-derived SMILES-embeddings.
- To enhance model performance and generalizability, particularly for small datasets.
- To provide insights into model predictions through atom contribution analysis.
Main Methods:
- Utilized a Transformer model to generate SMILES-embeddings from the internal encoder state.
- Employed a Convolutional Neural Network (CNN) architecture on these embeddings for QSAR/QSPR tasks.
- Implemented SMILES augmentation for both training and inference to improve robustness.
- Developed a standalone program for atom contribution analysis to interpret model results.
Main Results:
- Achieved higher quality interpretable QSAR/QSPR models on diverse benchmark datasets (regression and classification).
- Demonstrated effectiveness on small datasets due to transfer learning and augmentation based on embeddings.
- The Transformer-CNN method provides an internal consensus for prognosis.
- Atom contribution analysis enables interpretation of individual atom influences on predictions.
Conclusions:
- The Transformer-CNN method offers a powerful and interpretable approach for QSAR/QSPR modeling.
- The use of SMILES-embeddings facilitates effective transfer learning and data augmentation.
- The method's ability to handle small datasets and provide interpretability makes it valuable for cheminformatics and drug discovery.
- Open-source code and online implementation are available for broader accessibility.
