超越性能:设计选择如何塑造化学语言模型
Inken Fender1,2, Jannik Adrian Gut1,2, Thomas Lemmin3
1Institute of Biochemistry and Molecular Medicine, University of Bern, Bühlstrasse 28, 3012, Bern, Switzerland.
Journal of cheminformatics
|November 19, 2025
概括
化学语言模型 (CLMs) 在设计选择中显示了类似的性能,但内部表示显著不同. 原子智能代币化提高了可解释性,指导了化学接地CLM的发展.
科学领域:
- 计算化学计算化学
- 人工智能的人工智能
- 机器学习 机器学习
背景情况:
- 化学语言模型 (CLMs) 在分子性质预测和生成方面表现出色.
- 关键的设计选择,如分子表示,代币化和架构,对于它们对CLM可解释性的影响是未被充分探索的.
- 了解这些选择如何影响化学信息编码对于开发可靠的CLM至关重要.
研究的目的:
- 系统地评估分子表示,令牌化和架构对CLM性能和化学解释性的影响.
- 调查不同的设计选择如何影响CLM学到的内部表示.
- 为开发更有化学依据和可解释的CLM提供指导.
主要方法:
- 系统评估具有不同设计选择 (表示,代币化,架构) 的CLM.
- 微调下游任务上的CLM以评估性能.
- 使用探测预测器,矢量运算和维度减小技术探测潜伏空间结构.
主要成果:
- 在不同的模型配置中,下游任务性能在很大程度上是相似的.
- 基于设计选择,观察到内部表示的结构和可解释性的实质差异.
- 原子式代币化通常改善了CLM表示的化学解释性.
- 一个基于RoBERTa的模型与SMILES输入被证明是标准预测任务的可靠基线.
结论:
- 设计选择大大影响了CLM中化学信息的编码方式,影响了可解释性,而不是原始性能.
- 原子式代币化是提高CLM的化学解释性的一种有希望的策略.
- 目前的CLM设计选择,特别是带有SMILES的RoBERTa,提供了坚实的基础,但进一步的研究可以产生更易于解释的模型.
相关概念视频
Molecular Models
43.4K
Physical models representing molecular architectures of chemical compounds play essential roles in understanding chemistry. The use of molecular models makes it easier to visualize the structures and shapes of atoms and molecules.
43.4K
Language and Cognition
696
Language serves as a bridge between ideas and communication, influencing how individuals perceive and interact with the world. Psychologists have long debated whether language shapes thought or vice versa. This discussion gained grip with Edward Sapir and Benjamin Lee Whorf in the 1940s, who proposed that language determines thought, a concept known as linguistic determinism. They suggested that the vocabulary and structure of a language influence how its speakers think and perceive reality.
696
Polymer Classification: Stereospecificity
3.1K
Polymerization generates chiral centers along the entire backbone of a polymer chain. Accordingly, the stereochemistry of the substituent group has a significant effect on polymer properties. Polymers formed from monosubstituted alkene monomers feature chiral carbons at every alternate position in the polymer backbone. Relative to the predominant orientation of substituents at the adjacent chiral carbons, the polymer can exist in three different configurations: isotactic, syndiotactic, and...
3.1K
Molecular Shapes
61.2K
Molecules have characteristic shapes that are crucial for their function. The arrangement of various electron groups around the central atom dictates their molecular geometry. Electron pairs in the valence shell of a central atom will adopt an arrangement that minimizes repulsions between the electron pairs by maximizing the distance between them. The valence electrons form either bonding pairs, located primarily between bonded atoms, or lone pairs.
Two regions of electron density in a diatomic...
Two regions of electron density in a diatomic...
61.2K
Chemical Symbols
10.2K
A chemical symbol is an abbreviation that is used to indicate an element or an atom of an element. For example, the symbol for mercury is Hg. We use the same symbol to indicate one atom of mercury (microscopic domain) or to label a container of many atoms of the element mercury (macroscopic domain).
Some symbols are derived from the common name of the element; others are abbreviations of the name in another language. Most symbols have one or two letters, but three-letter symbols have been used...
Some symbols are derived from the common name of the element; others are abbreviations of the name in another language. Most symbols have one or two letters, but three-letter symbols have been used...
10.2K
Components of Language
732
Language, whether spoken, signed, or written, consists of specific components: lexicon and grammar. The lexicon is the vocabulary of a language, comprising its words. Grammar is the set of rules used to convey meaning through the lexicon. For example, English grammar adds “-ed” to most verbs to indicate past tense. Words are formed by combining phonemes, which are the basic sound units of a language. Different languages have different sets of phonemes (e.g., “ah” vs.
732


