逻辑:在立场检测中用于小型语言模型的内部认知改进的LLM起源的指导
Woojin Lee1, Jaewook Lee1, Harksoo Kim2
1Department of Artificial Intelligence, Konkuk University, Seoul, Republic of South Korea.
PeerJ. Computer science
|December 9, 2024
概括
本研究介绍了LOGIC,一种新的姿势检测模型,使用大型语言模型 (LLM) 来产生知识,提高准确性. 推理蒸增强了小语言模型 (SLM) 以实现高效,高性能立场检测.
科学领域:
- 自然语言处理自然语言处理.
- 计算社会科学 计算社会科学
背景情况:
- 姿态检测对于理解观点至关重要,但使用维基百科数据与小语言模型 (SLMs) 的传统方法面临质量和一致性问题.
- 大型语言模型 (LLM) 提供了卓越的知识生成,但是在资源有限的设备上部署的计算密集和难以部署.
研究的目的:
- 开发一种新的姿势检测模型,利用LLM产生的知识来提高准确性和效率.
- 通过引入SLM的推理蒸技术来解决LLM的部署挑战.
主要方法:
- 利用LLM来产生准确的,与上下文相关的目标知识,克服维基百科的数据限制.
- 引入了一个推理蒸方法来将LLM功能转移到SLMs.
- 开发了基于双向和自动回归变压器 (BART) 的LOGIC模型,与包括推理蒸在内的辅助学习任务进行了微调.
主要成果:
- 在VAried Stance Topics (VAST) 数据集中,LOGIC模型实现了最先进的性能.
- 在姿态检测任务中,LOGIC的表现优于GPT-3.5 Turbo和GPT-4 Turbo等先进型号.
- 推理蒸方法成功提高了SLM的效率,同时保持了强大的性能.
结论:
- 通过LLM生成的知识与推理蒸相结合,提供了一种强大的方法来增强立场检测.
- 逻辑模型为立场检测提供了一种高效和高性能解决方案,适用于更广泛的应用.
- 这项工作证明了将复杂的推理从LLM转移到更容易访问的SLM的潜力.
更多相关视频
相关概念视频
02:07Improving Translational Accuracy
9.0K
Base complementarity between the three base pairs of mRNA codon and the tRNA anticodon is not a failsafe mechanism. Inaccuracies can range from a single mismatch to no correct base pairing at all. The free energy difference between the correct and nearly correct base pairs can be as small as 3 kcal/ mol. With complementarity being the only proofreading step, the estimated error frequency would be one wrong amino acid in every 100 amino acids incorporated. However, error frequencies observed in...
9.0K
01:27Language and Cognition
323
Language serves as a bridge between ideas and communication, influencing how individuals perceive and interact with the world. Psychologists have long debated whether language shapes thought or vice versa. This discussion gained grip with Edward Sapir and Benjamin Lee Whorf in the 1940s, who proposed that language determines thought, a concept known as linguistic determinism. They suggested that the vocabulary and structure of a language influence how its speakers think and perceive reality.
323
01:12Types of Errors: Detection and Minimization
1.4K
Error is the deviation of the obtained result from the true, expected value or the estimated central value. Errors are expressed in absolute or relative terms.
Absolute error in a measurement is the numerical difference from the true or central value. Relative error is the ratio between absolute error and the true or central value, expressed as a percentage.
Errors can be classified by source, magnitude, and sign. There are three types of errors: systematic, random, and gross.
Systematic or...
Absolute error in a measurement is the numerical difference from the true or central value. Relative error is the ratio between absolute error and the true or central value, expressed as a percentage.
Errors can be classified by source, magnitude, and sign. There are three types of errors: systematic, random, and gross.
Systematic or...
1.4K
01:25The Anchoring-and-Adjustment Heuristic
7.2K
In order to make good decisions, we use our knowledge and our reasoning. Often, this knowledge and reasoning is sound and solid. However, sometimes, we are swayed by biases or by others manipulating a situation. For example, let’s say you and three friends wanted to rent a house and had a combined target budget of $1,600. The realtor shows you only very run-down houses for $1,600 and then shows you a very nice house for $2,000. Might you ask each person to pay more in rent to get the...
7.2K
02:16Stereotype Content Model
14.0K
The Stereotype Content Model (SCM) was first proposed by Susan Fiske and her colleagues (Fiske, Cuddy, Glick & Xu, 2002; see also Fiske, 2012 and Fiske, 2017). The SCM specifies that when someone encounters a new group, they will stereotype them based on two metrics: warmth—or that group’s perceived intent, and how likely they are to provide help or inflict harm—and competence—or their ability to carry out that objective. Depending on the warmth-competence...
14.0K