数学与语言模型:从记忆到计算
Davide Maltoni1, Matteo Ferrara1
1Department of Computer Science and Engineering, University of Bologna, Italy.
概括
大型语言模型可以执行算术计算,如二进制加法和乘法,通过将超越其训练数据的概括. 这些模型作为编码-回归-解码机用于计算任务.
科学领域:
- 人工智能的人工智能
- 计算语言学 计算语言学
- 机器学习 机器学习
背景情况:
- 最近的大型语言模型 (LLM) 展示了新兴的计算能力.
- 了解这些能力对于提高LLM性能和应用至关重要.
研究的目的:
- 调查在下一个令牌预测上训练的LLM如何执行算术计算.
- 分析LLM超越其训练数据的概括能力,用于数学任务.
主要方法:
- 在二进制加法和乘法任务上训练了一种轻量级的语言模型.
- 进行实验以评估外推能力和内部处理.
- 利用二进制算术作为一个测试台,因为它的小词汇和不连续性.
主要成果:
- 成功训练了一个语言模型来执行二进制加法和乘法.
- 证明语言模型可以将算术计算概括为新数据.
- 证据表明,在模型中,一个计算过程涉及编码,回归和解码.
结论:
- 语言模型可以被训练来执行算术计算与概括.
- 该模型似乎可以作为编码-回归-解码系统来完成这些任务.
- 在将输入令牌映射到内部表示后,计算发生在一个值空间中.
更多相关视频
05:48Author Spotlight: Investigating the Impact of Emotional Prosodies on Voice Recognition and Perception
Published on: August 9, 2024
1.5K
08:05Measuring Statistical Learning Across Modalities and Domains in School-Aged Children Via an Online Platform and Neuroimaging Techniques
Published on: June 30, 2020
7.5K
相关概念视频
Mnemonic Devices
69
Mnemonic devices are cognitive tools that facilitate memory retention by linking new information to familiar patterns or organizational strategies. These techniques are beneficial for remembering complex or lengthy sets of information by simplifying and structuring them in easily retrievable ways.
Acronyms
Acronyms are created by using the initial letters of a series of words to form a new word or phrase. This approach condenses complex information into a single, memorable entity. For example,...
Acronyms
Acronyms are created by using the initial letters of a series of words to form a new word or phrase. This approach condenses complex information into a single, memorable entity. For example,...
69
Mechanistic Models: Compartment Models in Algorithms for Numerical Problem Solving
47
Mechanistic models play a crucial role in algorithms for numerical problem-solving, particularly in nonlinear mixed effects modeling (NMEM). These models aim to minimize specific objective functions by evaluating various parameter estimates, leading to the development of systematic algorithms. In some cases, linearization techniques approximate the model using linear equations.
In individual population analyses, different algorithms are employed, such as Cauchy's method, which uses a...
In individual population analyses, different algorithms are employed, such as Cauchy's method, which uses a...
47
Elaborative Rehearsals
83
Elaborative rehearsal is a crucial cognitive strategy that strengthens information encoding in long-term memory by making meaningful connections between new data and pre-existing knowledge. This approach contrasts with maintenance rehearsal, which involves simple repetition without delving into the significance of the information. While maintenance rehearsal might temporarily keep information active in short-term memory, it is less effective for long-term retention.
The effectiveness of...
The effectiveness of...
83
Arithmetic Mean
13.6K
The arithmetic mean is the most commonly used measure of the central tendency of a data set. It is defined as the sum of all the elements constituting the data set, divided by the total number of elements. It is sometimes loosely referred to as the “average.”
When all the values in a data set are not unique, the sum in the numerator can be calculated by multiplying each distinct value by its frequency.
Sometimes, the arithmetic mean of a sample can be affected by a few data points...
When all the values in a data set are not unique, the sum in the numerator can be calculated by multiplying each distinct value by its frequency.
Sometimes, the arithmetic mean of a sample can be affected by a few data points...
13.6K
Implicit Memories
117
Implicit memories, also known as non-declarative memories, are long-term memories that function outside of conscious awareness. These memories influence behavior and skills without explicit knowledge. This type of memory is evident in tasks like playing tennis, snowboarding, and texting. Implicit memory has three subsystems: procedural memory, conditioning, and priming. This type of memory is essential in various activities, from everyday tasks to specialized skills.
One key aspect of implicit...
One key aspect of implicit...
117
Associative Learning
324
Associative learning is a fundamental concept in behavioral psychology, wherein a connection is established between two stimuli or events, leading to a learned response. This process is critical in understanding how behaviors are acquired and modified. Conditioning, the mechanism through which associations are formed, can be divided into two main types: classical conditioning and operant conditioning, each elucidating different aspects of associative learning.
Classical conditioning, also known...
Classical conditioning, also known...
324
