在一个情境敏感的随机语言模型中,Berezinskii-Kosterlitz-Thouless过渡
Yuma Toji1, Jun Takahashi2, Vwani Roychowdhury3
1Hokkaido University, Graduate School of Information Science and Technology, Sapporo, Hokkaido 060-0814, Japan.
Physical review. E
|February 20, 2026
概括
研究人员展示了一种新的语言模型中的相位过渡,灵感来自物理学. 这一发现表明,自然语言的关键性质可能源于Berezinskii-Kosterlitz-Thouless (BKT) 阶段,而不仅仅是微调.
科学领域:
- 统计物理 统计物理
- 计算语言学 计算语言学
- 自然语言处理自然语言处理.
背景情况:
- 自然语言中的权力定律关键性质类似于物理系统的相位过渡.
- 大型语言模型 (LLM) 与物理概念 (如缩放定律) 有相似之处.
- 在识别表现出统计物理定义相位过渡的生成语言模型方面存在差距.
研究的目的:
- 构建和分析一个展示相位过渡的概率语言模型.
- 调查统计物理概念对自然语言建模的适用性.
- 探索自然语言中关键性质的性质.
主要方法:
- 开发了一个情境敏感的随机语言模型,其灵感来源于1D Potts模型.
- 通过调整模型参数,通过数值证明了一个明确的相位过渡.
- 定义并分析了一个捕获符号频率偏差的顺序参数.
主要成果:
- 顺序参数在无限长度极限中从零过渡到非零,表示奇点.
- 确定过渡是贝雷津斯基-科斯特利茨-托勒斯 (BKT) 过渡的一个变体.
- 在一维系统中观察到BKT相位特征,这是一个罕见的现象.
结论:
- 自然语言的关键性质可以通过BKT阶段来解释.
- 这挑战了对微调或自我组织的批判性来解释语言属性的需求.
- 这项研究为1D系统中的BKT阶段提供了一个罕见的例子,对物理和语言学有影响.
相关概念视频
Woodward–Hoffmann Selection Rules and Microscopic Reversibility
Electrocyclic reactions, cycloadditions, and sigmatropic rearrangements are concerted pericyclic reactions that proceed via a cyclic transition state. These reactions are stereospecific and regioselective. The stereochemistry of the products depends on the symmetry characteristics of the interacting orbitals and the reaction conditions. Accordingly, pericyclic reactions are classified as either symmetry-allowed or symmetry-forbidden. Woodward and Hoffmann presented the selection criteria for...
Random Variables
A random variable is a single numerical value that indicates the outcome of a procedure. The concept of random variables is fundamental to the probability theory and was introduced by a Russian mathematician, Pafnuty Chebyshev, in the mid-nineteenth century.
Uppercase letters such as X or Y denote a random variable. Lowercase letters like x or y denote the value of a random variable. If X is a random variable, then X is written in words, and x is given as a number.
For example, let X = the...
Uppercase letters such as X or Y denote a random variable. Lowercase letters like x or y denote the value of a random variable. If X is a random variable, then X is written in words, and x is given as a number.
For example, let X = the...
Entropy Change in Reversible Processes
In the Carnot engine, which achieves the maximum efficiency between two reservoirs of fixed temperatures, the total change in entropy is zero. The observation can be generalized by considering any reversible cyclic process consisting of many Carnot cycles. Thus, it can be stated that the total entropy change of any ideal reversible cycle is zero.
The statement can be further generalized to prove that entropy is a state function. Take a cyclic process between any two points on a p-V diagram.
The statement can be further generalized to prove that entropy is a state function. Take a cyclic process between any two points on a p-V diagram.
Propagation of Uncertainty from Random Error
An experiment often consists of more than a single step. In this case, measurements at each step give rise to uncertainty. Because the measurements occur in successive steps, the uncertainty in one step necessarily contributes to that in the subsequent step. As we perform statistical analysis on these types of experiments, we must learn to account for the propagation of uncertainty from one step to the next. The propagation of uncertainty depends on the type of arithmetic operation performed on...
Transition State Theory
Transition-state theory, also known as activated-complex theory, provides a molecular-level explanation of reaction rates in both gas-phase and solution-phase reactions. It extends earlier kinetic models by considering the formation of a short-lived, high-energy configuration during a reaction.The progress of a chemical reaction can be represented using a reaction profile, which plots potential energy against the reaction coordinate. As two reactant molecules approach one another, their...
Language and Cognition
Language serves as a bridge between ideas and communication, influencing how individuals perceive and interact with the world. Psychologists have long debated whether language shapes thought or vice versa. This discussion gained grip with Edward Sapir and Benjamin Lee Whorf in the 1940s, who proposed that language determines thought, a concept known as linguistic determinism. They suggested that the vocabulary and structure of a language influence how its speakers think and perceive reality.

