一个因果关系意识的范式,用于评估多模式大语言模型的创造力
概括
评估多式联运大型语言模型 (LLM) 的创造力是一项挑战. 我们介绍了Oogiri游戏和LoTbench框架,用于全面的创造力评估,表明LLM创造力是可以改进的,并且与认知能力相关.
科学领域:
- 人工智能的人工智能
- 认知科学 认知科学
背景情况:
- 由于主观性和数据稀缺性,评估大型语言模型 (LLM) 的创意能力是很困难的,特别是在多式联络环境中.
- 现有的基准主要集中在逻辑推理上,忽视了人工智能发展至关重要的创造性方面.
研究的目的:
- 开发一个全面的管道来评估多式联络法学士的创造力.
- 引入新的平台和方法来评估超越传统指标的AI创造力.
主要方法:
- 作为一个评估平台,利用了Oogiri游戏,这是一项以创造力为重点的任务,涉及幽默和关联思维.
- 提出LoTbench,一个交互式,因果关系意识的框架来量化创造力和可视化思维过程,减轻信息泄露等风险.
- 将LoTbench的结果与传统的创造力指标和多式模式认知基准MMMU进行了比较.
主要成果:
- 确定了Oogiri游戏作为一个适合多式模式LLM创意评估的平台.
- LoTbench有效地量化了LLM的创造力,并可视化了潜在的认知过程.
- 观察到LLM和人类之间的性能差距,但发现它不是无法克服的.
- 发现LoTbench和MMMU之间存在强烈的相关性,这表明它们与人类的认知创造力理论保持一致.
结论:
- 多模式法学士创意评估需要专门的平台和框架,如Oogiri和LoTbench.
- 根据认知原则,LoTbench提供了一种更有效和更易于解释的方法来评估AI创造力.
- 认知在早期的创造力中起着至关重要的作用,使LLMs的概念桥梁和多样化的想法产生成为可能.
相关概念视频
01:27Language and Cognition
313
Language serves as a bridge between ideas and communication, influencing how individuals perceive and interact with the world. Psychologists have long debated whether language shapes thought or vice versa. This discussion gained grip with Edward Sapir and Benjamin Lee Whorf in the 1940s, who proposed that language determines thought, a concept known as linguistic determinism. They suggested that the vocabulary and structure of a language influence how its speakers think and perceive reality.
313
02:16Stereotype Content Model
13.9K
The Stereotype Content Model (SCM) was first proposed by Susan Fiske and her colleagues (Fiske, Cuddy, Glick & Xu, 2002; see also Fiske, 2012 and Fiske, 2017). The SCM specifies that when someone encounters a new group, they will stereotype them based on two metrics: warmth—or that group’s perceived intent, and how likely they are to provide help or inflict harm—and competence—or their ability to carry out that objective. Depending on the warmth-competence...
13.9K
01:10Higher Mental Functions of the Brain: Language
707
Language is a system of communication that allows the expression of thoughts, ideas, and feelings. The brain processes language in both hemispheres.
Language formation and comprehension take place in the dominant hemisphere. The dominant hemisphere is responsible for understanding the meaning of spoken, written, or sign language, as well as the ability to communicate. For most people, the left hemisphere is the dominant one. The right hemisphere, then, gives tone and emotional context to the...
Language formation and comprehension take place in the dominant hemisphere. The dominant hemisphere is responsible for understanding the meaning of spoken, written, or sign language, as well as the ability to communicate. For most people, the left hemisphere is the dominant one. The right hemisphere, then, gives tone and emotional context to the...
707
01:28Lateralization
299
Brain lateralization refers to the division of mental processes and functions between the two hemispheres of the brain, a phenomenon that optimizes neural efficiency and underpins complex abilities in humans. This specialization allows each hemisphere to perform tasks where it has a comparative advantage, facilitating more refined cognitive capabilities across different domains.
299
01:21Cognitive Learning
144
Cognitive learning is based on purposive behavior, incidental learning, and insight learning.
E. C. Tolman's theory of purposive behavior emphasizes that much behavior is goal-directed. He argued that to understand behavior, we must look at the entire sequence of actions leading to a goal. For instance, high school students study hard, not just due to past reinforcement but also to achieve the goal of getting into a good college.
Tolman introduced the idea that behavior is influenced by...
E. C. Tolman's theory of purposive behavior emphasizes that much behavior is goal-directed. He argued that to understand behavior, we must look at the entire sequence of actions leading to a goal. For instance, high school students study hard, not just due to past reinforcement but also to achieve the goal of getting into a good college.
Tolman introduced the idea that behavior is influenced by...
144
01:20Multiple Intelligences Theory
6.2K
Howard Gardner's theory of Multiple Intelligence proposes that there are nine distinct types of intelligence, each reflecting different ways of interacting with the world. Introduced in 1983 and expanded in subsequent years, Gardner's framework challenges the traditional notion of a single, generalized intelligence.
6.2K