智慧的阴影:通过大型语言模型对元认知和道德基础的叙事内容进行分类
Alexander Stavropoulos1, Damien L Crone2, Igor Grossmann3
1Department of Psychology, University of Waterloo, Waterloo, N2L 3G1, Canada.
Behavior research methods
|May 29, 2024
概括
大型语言模型 (LLM) 在冲突叙述中有效地分类复杂的心理构造,显示与人类编码器的高度一致. 这项技术有望通过自动化复杂的编码任务来改变社会科学研究.
科学领域:
- 心理学 心理学 心理学
- 人工智能的人工智能
- 计算社会科学 计算社会科学
背景情况:
- 将复杂的心理结构归类为定性数据是劳动密集型和主观的.
- 大型语言模型 (LLM) 提供了在社会科学中自动化文本分析的潜力.
- 之前的研究还没有广泛评估LLMs的细微心理构造分类.
研究的目的:
- 评估最先进的LLM在分类复杂心理构造中的有效性.
- 在现实世界的冲突场景中,将LLM绩效与专业的人类编码器进行比较.
- 探索LLMs在心理学研究中自动化定性数据编码的潜力.
主要方法:
- 使用GPT-4和RoB-ELoC (RoBERTa-微调情感与逻辑回归分类器) 进行分类.
- 采用了几次射击和零次射击的学习模式.
- 分析了347名加拿大和美国成年人反映工作场所冲突的叙述.
- 通过使用协议指标和科恩的卡帕,与专家人类编码人员比较了LLM分类.
主要成果:
- LLM表现出强大的分类表现,达到了80%以上的一致性,F1得分高于0.85.
- 观察到人类模型的可靠性很高,科恩的卡帕 (κ) 在顶级模型中为0.80.
- 特别有效的是RoB-ELoC和GPT-4,尽管对智力谦卑的影响较小.
- 提供了研究整合工作流程的示例.
结论:
- LLM 显示出对自动化复杂心理构造的编码具有显著的可行性.
- 开源和商业LLM都可以成为社会科学研究中的有价值的工具.
- 这种方法有可能显著改变心理学和相关领域的定性数据分析.
更多相关视频
12:55Multimodal Protocol for Assessing Metacognition and Self-Regulation in Adults with Learning Difficulties
Published on: September 27, 2020
8.4K
08:05Measuring Statistical Learning Across Modalities and Domains in School-Aged Children Via an Online Platform and Neuroimaging Techniques
Published on: June 30, 2020
7.5K
相关概念视频
Metacognition
150
Metacognition is a conscious process where individuals are aware of their cognitive and executive processes, such as planning before solving a problem or self-monitoring during reading. For instance, a writer may need help with composing a piece. The situation involves a writer who is working on a piece of writing, but while doing so, they realize that something is missing. They notice that their characters lack depth or details. This realization occurs because the writer is reflecting on their...
150
Stereotype Content Model
14.7K
The Stereotype Content Model (SCM) was first proposed by Susan Fiske and her colleagues (Fiske, Cuddy, Glick & Xu, 2002; see also Fiske, 2012 and Fiske, 2017). The SCM specifies that when someone encounters a new group, they will stereotype them based on two metrics: warmth—or that group’s perceived intent, and how likely they are to provide help or inflict harm—and competence—or their ability to carry out that objective. Depending on the warmth-competence...
14.7K
Higher Mental Functions of the Brain: Language
793
Language is a system of communication that allows the expression of thoughts, ideas, and feelings. The brain processes language in both hemispheres.
Language formation and comprehension take place in the dominant hemisphere. The dominant hemisphere is responsible for understanding the meaning of spoken, written, or sign language, as well as the ability to communicate. For most people, the left hemisphere is the dominant one. The right hemisphere, then, gives tone and emotional context to the...
Language formation and comprehension take place in the dominant hemisphere. The dominant hemisphere is responsible for understanding the meaning of spoken, written, or sign language, as well as the ability to communicate. For most people, the left hemisphere is the dominant one. The right hemisphere, then, gives tone and emotional context to the...
793
Cognitivism
1.4K
Cognitive psychology emerged as a significant field in the mid-20th century. It focused on understanding humans' internal mental processes. This approach emphasizes how people perceive, remember, think, and solve problems—elements critical to human cognition.
Previously dominated by behaviorism, which prioritized observable behaviors and largely ignored mental processes, psychology transformed in the 1950s. Cognitive psychologists argue that understanding how we think and process...
Previously dominated by behaviorism, which prioritized observable behaviors and largely ignored mental processes, psychology transformed in the 1950s. Cognitive psychologists argue that understanding how we think and process...
1.4K
Language and Cognition
342
Language serves as a bridge between ideas and communication, influencing how individuals perceive and interact with the world. Psychologists have long debated whether language shapes thought or vice versa. This discussion gained grip with Edward Sapir and Benjamin Lee Whorf in the 1940s, who proposed that language determines thought, a concept known as linguistic determinism. They suggested that the vocabulary and structure of a language influence how its speakers think and perceive reality.
342
Deductive Reasoning
55.2K
Deductive reasoning, or deduction, is the type of logic used in hypothesis-based science. In deductive reasoning, the pattern of thinking moves in the opposite direction as compared to inductive reasoning, which means that it uses a general principle or law to predict specific results. From those general principles, a scientist can deduce and predict the specific results that would be valid as long as the general principles are valid.
For example, a researcher can deduce specific predictions...
For example, a researcher can deduce specific predictions...
55.2K
