自动回归的实例表明,大型语言模型是如何被训练来解决的问题所塑造的
R Thomas McCoy1, Shunyu Yao1, Dan Friedman1
1Department of Computer Science, Princeton University, Princeton, NJ 08542.
概括
大型语言模型 (LLM) 的成功或失败取决于其培训目标:下一个词的预测. 了解这种远程方法揭示了概率如何影响LLM准确性,并确定了潜在的故障模式.
科学领域:
- 人工智能的人工智能
- 自然语言处理自然语言处理.
- 机器学习 机器学习
背景情况:
- 大型语言模型 (LLM) 越来越多地被采用,需要清楚地了解它们的功能和局限性.
- 当前的评估往往将LLM人类化,可能会忽视它们独特的操作特征.
研究的目的:
- 为了解LLM行为提出一个"teleological方法",专注于他们的核心培训目标:下一个词的预测.
- 根据这种方法确定影响LLM准确性的关键因素,并经验测试这些假设.
主要方法:
- 在11个不同的任务中评估了五个突出的LLM (GPT-3.5,GPT-4,Claude 3,Llama 3,Gemini 1.0).
- 研究了输入概率,任务概率和输出概率对LLM绩效的影响.
- 分析故障模式,特别是在理论上不应该考虑概率因素的决定性任务中.
主要成果:
- 经验证据证实,LLM准确性受到任务,输入和输出的概率的显著影响,正如假设的那样.
- 发现了令人惊的故障模式,比如GPT-4在确定性加密任务中的精度降低,当预期输出的可能性低时.
- 证明LLM表现出与其培训固有的概率因素直接相关的性能变化.
结论:
- 应将LLM视为由其培训压力塑造的独特系统,而不是像人类一样的智能.
- 人工智能从业者在涉及低概率结果的场景中部署LLM时必须谨慎.
- 远程学方法为预测和理解LLM行为和局限性提供了一个框架.
更多相关视频
09:09Foreign Accent and Forensic Speaker Identification in Voice Lineups: The Influence of Acoustic Features Based on Prosody
Published on: September 27, 2024
411
06:48Lexical Decision Task for Studying Written Word Recognition in Adults with and without Dementia or Mild Cognitive Impairment
Published on: June 25, 2019
9.1K
相关概念视频
Improving Translational Accuracy
9.3K
Base complementarity between the three base pairs of mRNA codon and the tRNA anticodon is not a failsafe mechanism. Inaccuracies can range from a single mismatch to no correct base pairing at all. The free energy difference between the correct and nearly correct base pairs can be as small as 3 kcal/ mol. With complementarity being the only proofreading step, the estimated error frequency would be one wrong amino acid in every 100 amino acids incorporated. However, error frequencies observed in...
9.3K
Multiple Regression
3.0K
Multiple regression assesses a linear relationship between one response or dependent variable and two or more independent variables. It has many practical applications.
Farmers can use multiple regression to determine the crop yield based on more than one factor, such as water availability, fertilizer, soil properties, etc. Here, the crop yield is the response or dependent variable as it depends on the other independent variables. The analysis requires the construction of a scatter plot...
Farmers can use multiple regression to determine the crop yield based on more than one factor, such as water availability, fertilizer, soil properties, etc. Here, the crop yield is the response or dependent variable as it depends on the other independent variables. The analysis requires the construction of a scatter plot...
3.0K
Regression Toward the Mean
6.3K
Regression toward the mean (“RTM”) is a phenomenon in which extremely high or low values—for example, and individual’s blood pressure at a particular moment—appear closer to a group’s average upon remeasuring. Although this statistical peculiarity is the result of random error and chance, it has been problematic across various medical, scientific, financial and psychological applications. In particular, RTM, if not taken into account, can interfere when...
6.3K
Regression Analysis
5.6K
Regression analysis is a statistical tool that describes a mathematical relationship between a dependent variable and one or more independent variables.
In regression analysis, a regression equation is determined based on the line of best fit– a line that best fits the data points plotted in a graph. This line is also called the regression line. The algebraic equation for the regression line is called the regression equation. It is represented as:
In regression analysis, a regression equation is determined based on the line of best fit– a line that best fits the data points plotted in a graph. This line is also called the regression line. The algebraic equation for the regression line is called the regression equation. It is represented as:
5.6K
Language and Cognition
335
Language serves as a bridge between ideas and communication, influencing how individuals perceive and interact with the world. Psychologists have long debated whether language shapes thought or vice versa. This discussion gained grip with Edward Sapir and Benjamin Lee Whorf in the 1940s, who proposed that language determines thought, a concept known as linguistic determinism. They suggested that the vocabulary and structure of a language influence how its speakers think and perceive reality.
335
Truncation in Survival Analysis
173
Truncation in survival analysis refers to the exclusion of individuals or events from the dataset based on specific criteria related to the time of the event. This exclusion can happen in two primary forms: left truncation and right truncation.
Left truncation occurs when individuals who experienced the event of interest before a certain time are not included in the study. This is often due to a "delayed entry" into the study where only those who survive until a certain entry point are...
Left truncation occurs when individuals who experienced the event of interest before a certain time are not included in the study. This is often due to a "delayed entry" into the study where only those who survive until a certain entry point are...
173
