评分德语 替代使用项目 应用大型语言模型
Janika Saretzki1,2,3, Thomas Knopf4, Boris Forthmann5
1Department of Psychology, University of Graz, 8010 Graz, Austria.
Journal of Intelligence
|June 25, 2025
概括
大型语言模型 (LLM) 显示出在德国替代使用任务 (AUT) 中评分创意潜力的希望. OCSAI和GPT-4表现最好,翻译结果并没有持续改善.
科学领域:
- 心理学 心理学 心理学
- 人工智能的人工智能
- 认知科学 认知科学
背景情况:
- 替代用途任务 (AUT) 是评估创意潜力的关键措施.
- 人类对AUT响应的评分是资源密集和耗时的.
- 大型语言模型 (LLM) 为自动化AUT评分提供了一个潜在的解决方案.
研究的目的:
- 为了评估不同LLM对得分德国AUT答案的有效性.
- 将微调的多语言LLM方法 (CLAUS,OCSAI) 与GPT-4进行比较.
- 评估跨不同数据集和条件的LLM绩效的概括性.
主要方法:
- 编制了来自多项研究的约50,000个德国AUT响应的大数据集.
- 与OCSAI,CLAUS和GPT-4进行比较,以评分原始的德语答案和英语翻译.
- 使用预先注册的分析计划来确保方法的严格性.
主要成果:
- 基于LLM的评分显示了与人类评分的实质性相关性.
- OCSAI显示了最高的相关性,其次是GPT-4和CLAUS.
- 将答案翻译成英语并没有始终提高得分准确度.
结论:
- LLM,特别是OCSAI和GPT-4,是德国AUT答案自动评分的有效工具.
- 选择的LLM影响得分准确性,OCSAI和GPT-4是强有力的候选人.
- 进一步的研究应该探索可通用性和优化LLM应用程序的创造力评估.
关键词:
在 GPT 中,GPT 必须是 GPT.德国 德国人 德国人 德国人 德国人另类使用任务任务.评估评估的评估评估的评估.自动评分的自动化评分.创造力 创造力 创造力不同的思维,不同的思维.大型语言模型.更多相关视频
06:48Lexical Decision Task for Studying Written Word Recognition in Adults with and without Dementia or Mild Cognitive Impairment
Published on: June 25, 2019
9.3K
12:49Transcranial Direct Current Stimulation tDCS of Wernicke's and Broca's Areas in Studies of Language Learning and Word Acquisition
Published on: July 13, 2019
17.3K
相关概念视频
Components of Language
413
Language, whether spoken, signed, or written, consists of specific components: lexicon and grammar. The lexicon is the vocabulary of a language, comprising its words. Grammar is the set of rules used to convey meaning through the lexicon. For example, English grammar adds “-ed” to most verbs to indicate past tense. Words are formed by combining phonemes, which are the basic sound units of a language. Different languages have different sets of phonemes (e.g., “ah” vs.
413
Language and Cognition
460
Language serves as a bridge between ideas and communication, influencing how individuals perceive and interact with the world. Psychologists have long debated whether language shapes thought or vice versa. This discussion gained grip with Edward Sapir and Benjamin Lee Whorf in the 1940s, who proposed that language determines thought, a concept known as linguistic determinism. They suggested that the vocabulary and structure of a language influence how its speakers think and perceive reality.
460
Language Development
461
Children master language quickly and with relative ease, supported by both biological predisposition and reinforcement. B. F. Skinner (1957) proposed that language is learned through reinforcement, while Noam Chomsky (1965) argued that language acquisition mechanisms are biologically determined.
The critical period for language acquisition suggests that the ability to acquire language is at its peak early in life. As people age, this proficiency decreases. Language development begins very...
The critical period for language acquisition suggests that the ability to acquire language is at its peak early in life. As people age, this proficiency decreases. Language development begins very...
461
Language
430
Language is a unique communication system that uses words and systematic rules to organize and transmit information. Unlike other forms of communication, which may involve postures, movements, odors, or vocalizations, language relies on symbols and grammar. This makes human communication distinct from that of other species, who also communicate but do not use language in the same way humans do.
Corballis and Suddendorf (2007) and Tomasello and Rakoczy (2003) highlight the role of language in...
Corballis and Suddendorf (2007) and Tomasello and Rakoczy (2003) highlight the role of language in...
430
Improving Translational Accuracy
11.9K
Base complementarity between the three base pairs of mRNA codon and the tRNA anticodon is not a failsafe mechanism. Inaccuracies can range from a single mismatch to no correct base pairing at all. The free energy difference between the correct and nearly correct base pairs can be as small as 3 kcal/ mol. With complementarity being the only proofreading step, the estimated error frequency would be one wrong amino acid in every 100 amino acids incorporated. However, error frequencies observed in...
11.9K
Introduction to z Scores
10.0K
A z score (or standardized value) is measured in units of the standard deviation. It tells you how many standard deviations the value x is above (to the right of) or below (to the left of) the mean, μ. Values of x that are larger than the mean have positive z scores, and values of x that are smaller than the mean have negative z scores. If x equals the mean, then x has a zero z score. It is important to note that the mean of the z scores is zero, and the standard deviation is one.
z scores...
z scores...
10.0K
