评估LLM's的分歧思维能力,以产生科学想法,并提供最少的背景
Kai Ruan1, Xuan Wang2, Jixiang Hong1
1Gaoling School of Artificial Intelligence, Renmin University of China, Beijing, China.
Nature communications
|March 7, 2026
概括
大型语言模型 (LLM) 在科学中表现有前途,但当前的测试错过了他们的创意生成技能. 一个新的基准,LiveIdeaBench,揭示了LLM创造力不是很好地预测一般智力分数.
科学领域:
- 人工智能的人工智能
- 计算科学 计算科学
- 科学发现 科学发现 发现
背景情况:
- 大型语言模型 (LLM) 在文学分析和实验设计等科学任务中表现出色.
- 现有的基准通常使用丰富的上下文输入,可能会忽视特定的创意能力.
- 评估LLMs在科学想法生成的不同思维仍然是一个挑战.
研究的目的:
- 介绍LiveIdeaBench,这是一个评估LLM科学想法产生的新基准.
- 评估LLM's使用单个关键字提示的分歧思维,灵感来自创造力理论.
- 评估一般智能分数与科学想法生成能力之间的关系.
主要方法:
- 开发了LiveIdeaBench,这是一个使用22个科学领域的1180个关键字的基准.
- 雇佣了一个由40多名最先进的法学士组成的动态小组.
- 通过五个方面评估产生的想法:原创性,可行性,流性,灵活性和清晰度.
主要成果:
- 通过标准的一般情报指标,LLM的科学想法生成能力的预测很差.
- 一般智力得分较低的模型可以与得分较高的模型展现相似的创造性表现.
- 在不同的LLMs中,创造性创意产生存在显著差异.
结论:
- 目前的评估方法可能无法充分利用LLM的科学创造力潜力.
- 像LiveIdeaBench这样的专业基准对于理解和改进LLM科学想法产生至关重要.
- 开发用于科学想法生成的LLM可能需要与一般问题解决相比,不同的培训策略.
更多相关视频
10:26Problem-Solving Before Instruction PS-I: A Protocol for Assessment and Intervention in Students with Different Abilities
Published on: September 11, 2021
4.5K
06:45Task Interruption and Resumption Paradigm for Testing the Activation and Pursuit of an Abstract Thinking Goal
Published on: April 18, 2017
6.7K
相关概念视频
Creative Thinking
1.4K
Creative thinking encompasses innovative and unconventional methods for addressing challenges, often leading to groundbreaking solutions. Instead of focusing solely on enhancing existing systems, such as increasing smartphone battery capacity, creative thinking might inspire advancements like energy-efficient batteries or processors that minimize power consumption. This multidimensional approach underscores the importance of exploring novel pathways to innovation.
Divergent thinking is the...
Divergent thinking is the...
1.4K
Cognitive Learning
1.5K
Cognitive learning is based on purposive behavior, incidental learning, and insight learning.
E. C. Tolman's theory of purposive behavior emphasizes that much behavior is goal-directed. He argued that to understand behavior, we must look at the entire sequence of actions leading to a goal. For instance, high school students study hard, not just due to past reinforcement but also to achieve the goal of getting into a good college.
Tolman introduced the idea that behavior is influenced by...
E. C. Tolman's theory of purposive behavior emphasizes that much behavior is goal-directed. He argued that to understand behavior, we must look at the entire sequence of actions leading to a goal. For instance, high school students study hard, not just due to past reinforcement but also to achieve the goal of getting into a good college.
Tolman introduced the idea that behavior is influenced by...
1.5K
Counterfactual Thinking
315
Counterfactual thinking is a cognitive process wherein individuals mentally reconstruct alternative versions of past events, often beginning with “what if” or “if only.” This reflective mechanism plays a significant role in shaping emotional experiences and guiding future behavior. Though typically triggered by unfavorable or unexpected outcomes, counterfactual thinking can also emerge in mundane, everyday decisions and experiences, revealing its deep entrenchment in...
315
Inductive Reasoning
68.9K
Inductive reasoning is a form of logical thinking that uses related observations to arrive at a general conclusion. It is uncertain and operates in degrees to which the conclusions are credible. As such, inductive arguments can be weak or strong, rather than valid or invalid, and conclusions can be used to formulate testable, falsifiable hypotheses.
Inductive reasoning is common in descriptive science. A life scientist makes observations and records them. This data can be qualitative or...
Inductive reasoning is common in descriptive science. A life scientist makes observations and records them. This data can be qualitative or...
68.9K
Ampere-Maxwell's Law: Problem-Solving
1.3K
A parallel-plate capacitor with capacitance C, whose plates have area A and separation distance d, is connected to a resistor R and a battery of voltage V. The current starts to flow at t = 0. What is the displacement current between the capacitor plates at time t? From the properties of the capacitor, what is the corresponding real current?
To solve the problem, we can use the equations from the analysis of an RC circuit and Maxwell's version of Ampère's law.
For the first part of the...
To solve the problem, we can use the equations from the analysis of an RC circuit and Maxwell's version of Ampère's law.
For the first part of the...
1.3K
High-Level and Low-Level Awareness
863
Controlled processes in human consciousness represent high-alert mental states where individuals deliberately focus their attention on achieving specific goals. Controlled processes can be seen in situations like mastering new technology, where a person might become so absorbed that they ignore surrounding distractions. Such processes involve selective attention, requiring one to concentrate on particular elements of experience while disregarding others. These are governed by executive...
863
