坚持自己的角色! 在大型语言模型中表达的个人价值观的稳定性
Grgur Kovač1, Rémy Portelas1,2, Masataka Sawayama3
1Flowers Team, INRIA, Bordeaux, France.
PloS one
|August 26, 2024
概括
大型语言模型 (LLM) 在不同环境中显示不同的价值稳定性. 一些LLM家族比其他更稳定,特别是当不模拟人格时,突出需要更好的上下文依赖研究.
科学领域:
- 人工智能的人工智能
- 计算语言学 计算语言学
- 心理学 心理学 心理学
背景情况:
- 目前的大型语言模型 (LLM) 的评估通常使用最小的上下文,这可能不会反映现实世界的部署,因为LLM的上下文依赖性.
- 语境依赖性是对LLM与认知能力和模型大小进行比较的关键,但尚未研究的维度.
研究的目的:
- 调查21个LLM在各种对话环境中的价值表达的稳定性.
- 通过心理和行为测量,比较LLM稳定性与和没有模拟个性.
- 建立一个关于LLM价值稳定性和上下文依赖性的基础研究.
主要方法:
- 通过使用标准心理学问卷 (PVQ) 和下游行为任务,评估了来自六个家庭的21个LLM.
- 在不同主题的模拟对话中评估了等级顺序 (人际关系) 和Ipsative (人际关系) 稳定性.
- 在具有和没有受指导的人格模拟的环境中比较模型性能.
主要成果:
- 与LLaMa-2和Phi.相比,Mixtral,Mistral,GPT-3.5和Qwen家族的价值稳定性更高.
- 稳定性趋势在不同的模拟群体和下游任务中是一致的.
- 被指导的人格模拟导致了较低的等级顺序稳定性,随着更长的对话,这种稳定性进一步下降.
结论:
- 在LLM中,价值表达的稳定性在模型家族之间有很大差异,并受到环境和人格模拟的影响.
- 目前用于评估LLM的方法可能无法完全捕捉其在动态,现实世界的环境中的行为.
- 未来的研究应该专注于开发方法,以更有效地研究LLM的上下文依赖性和人格连贯性.
更多相关视频
相关概念视频
01:14Stability of structures
157
In mechanical engineering, the stability of systems under various forces is critical for designing durable and efficient structures. One fundamental way to explore these concepts is by analyzing systems like two rods connected at a pivot point, O, with a torsional spring of spring constant k at the pivot point. This system is similar in appearance to a scissor jack used to change tires on a car. In this case, the arms of the linkage (equivalent to the rods in this system) are entirely vertical,...
157
01:27Language and Cognition
339
Language serves as a bridge between ideas and communication, influencing how individuals perceive and interact with the world. Psychologists have long debated whether language shapes thought or vice versa. This discussion gained grip with Edward Sapir and Benjamin Lee Whorf in the 1940s, who proposed that language determines thought, a concept known as linguistic determinism. They suggested that the vocabulary and structure of a language influence how its speakers think and perceive reality.
339
02:16Stereotype Content Model
14.0K
The Stereotype Content Model (SCM) was first proposed by Susan Fiske and her colleagues (Fiske, Cuddy, Glick & Xu, 2002; see also Fiske, 2012 and Fiske, 2017). The SCM specifies that when someone encounters a new group, they will stereotype them based on two metrics: warmth—or that group’s perceived intent, and how likely they are to provide help or inflict harm—and competence—or their ability to carry out that objective. Depending on the warmth-competence...
14.0K
01:28Stability
97
The time response of a linear time-invariant (LTI) system can be divided into transient and steady-state responses. The transient response represents the system's initial reaction to a change in input and diminishes to zero over time. In contrast, the steady-state response is the behavior that persists after the transient effects have faded.
The stability of an LTI system is determined by the roots of its characteristic equation, known as poles. A system is stable if it produces a bounded...
The stability of an LTI system is determined by the roots of its characteristic equation, known as poles. A system is stable if it produces a bounded...
97
01:29Professional Values
6.5K
Nurses are responsible for caring for patients during birth, death, illness, and healing. Professional values guide the decisions and actions that nurses make in their careers. If nurses know the decisions and actions to take, providing patients with exceptional care is possible.
The values that are the foundation of the nursing profession are altruism, autonomy, human dignity, and social justice.
First, altruism refers to the concern for the welfare and well-being of others without personal...
The values that are the foundation of the nursing profession are altruism, autonomy, human dignity, and social justice.
First, altruism refers to the concern for the welfare and well-being of others without personal...
6.5K
01:26Constraints and Statical Determinacy
592
In structural engineering, the equilibrium of a system is not only determined by its equations of equilibrium but also with the help of constraints. Constraints refer to restrictions on the motion of a system. The proper combinations of constraints can minimize the total number of constraints needed to maintain a system in mechanical equilibrium. When this happens, the system is said to be statically determinate. For such systems, the unknown reaction supports can be estimated using equilibrium...
592