使用BERT进行分子性质预测的定位嵌入和零射击学习
Medard Edmund Mswahili1, JunHa Hwang1, Jagath C Rajapakse2
1Department of Computer Engineering, Chungbuk National University, Cheongju, 28644, South Korea.
Journal of cheminformatics
|February 5, 2025
概括
这项研究通过优化化学简化分子输入线输入系统 (SMILES) 数据的位置编码 (PE) 来提高使用变压器模型的分子性质预测. 这项研究表明,在预测未见分子表征的属性方面,准确性和概括性得到了提高.
科学领域:
- 化学信息学 化学信息学
- 深度学习 (Deep Learning) 是一种深度学习.
- 自然语言处理自然语言处理.
背景情况:
- 化学信息学的进步需要改进化学简化分子输入线输入系统 (SMILES) 数据的处理.
- 变压器模型在自然语言处理方面取得了成功,对分子数据分析有希望.
- 定位编码 (PE) 对变压器模型来捕获分子表示中的序列和上下文信息至关重要.
研究的目的:
- 在基于变压器的框架内探索和优化各种位置编码 (PE) 以提高分子性质预测.
- 研究PE对使用SMILES和DeepSMILES进行化学文本分析的变压器 (BERT) 模型的双向编码器表示的准确性和概括性的影响.
- 评估不同PE的BERT模型在不同分子数据集上的零射击学习能力.
主要方法:
- 在SMILES字符串上预训练来自变压器 (BERT) 模型的双向编码器表示,具有各种位置编码 (PE).
- 在下游分子性质预测任务上微调表现最好的BERT模型.
- 利用SMILES和DeepSMILES的表征来对现有和新提议的数据集进行全面评估.
主要成果:
- 通过BERT模型中的优化PE,在分子性质预测中证明了更好的准确性和概括性.
- 成功地将BERT模型与各种PE应用到各种数据集,包括COVID-19和生物试验数据.
- 展示了模型在预测未见分子表示的属性的零射击学习中的熟练程度.
结论:
- 定位编码显著提高了像BERT这样的变压器模型的性能,用于分子性质预测.
- 当配备适当的PE时,BERT模型表现出强度和在化学信息学和生物信息学中广泛应用的潜力.
- 该研究强调了PE在捕获SMILES字符串中的复杂原子关系的有效性,以进行准确的化学数据分析.
相关概念视频
03:51Molecular Orbital Theory II
18.9K
Molecular Orbital Energy Diagrams
18.9K
02:27Predicting Molecular Geometry
34.0K
VSEPR Theory for Determination of Electron Pair Geometries
34.0K
02:40Ligand Binding Sites
12.7K
Proteins are dynamic macromolecules that carry out a wide variety of essential processes; however, the activities of most proteins depend on their interactions with other molecules or ions, known as ligands.
Protein-ligand interactions are quite specific; even though numerous potential ligands surround a cellular protein at any given time, only a particular ligand can bind to that protein. Moreover, a ligand binds only to a dedicated area on the surface of the protein, known as the...
Protein-ligand interactions are quite specific; even though numerous potential ligands surround a cellular protein at any given time, only a particular ligand can bind to that protein. Moreover, a ligand binds only to a dedicated area on the surface of the protein, known as the...
12.7K
01:18Structure of Benzene: Molecular Orbital Model
8.7K
According to the molecular orbital (MO) model, benzene has a planar structure with a regular hexagon of six sp2 hybridized carbons. As shown in Figure 1, each carbon is bonded to three other atoms with C–C–C and H–C–C bond angles of 120°. The C–H bond length is 109 pm, and the C–C bond length is 139 pm which is midway between the single bond length of sp3 hybridized carbons (154 pm) and sp2 hybridized carbons (133 pm).
8.7K
03:24Hybridization of Atomic Orbitals I
46.4K
The mathematical expression known as the wave function, ψ, contains information about each orbital and the wavelike properties of electrons in an isolated atom. When atoms are bound together in a molecule, the wave functions combine to produce new mathematical descriptions that have different shapes. This process of combining the wave functions for atomic orbitals is called hybridization and is mathematically accomplished by the linear combination of atomic orbitals. The new orbitals that...
46.4K
01:27π Electron Effects on Chemical Shift: Overview
1.0K
An applied magnetic field causes loosely bound π-electrons in organic molecules to circulate, producing a local or induced diamagnetic field over a large spatial volume. As the molecules tumble in solution, the field generated by π-electrons in spherical substituents results in a zero net field. However, the net field generated by π-electrons in non-spherical substituents is not zero. The effect of this induced field depends on the orientation of the molecule with respect to B0,...
1.0K