生成型人工智能可以从语言中推断出思维方式吗? 评估AI作为心理文本分析工具的实用性
1Department of Communication, Michigan State University, East Lansing, MI, USA. dmm@msu.edu.
Behavior research methods
|January 26, 2024
概括
生成型人工智能 (AI) 对文本分析的效用有限,在ChatGPT和语言查询和词汇计数 (LIWC) 分析思维得分之间发现了小的关联. 在这里,我们可以看到AIAIAI.
科学领域:
- 计算语言学 计算语言学
- 人工智能的人工智能
- 心理测量 心理测量 心理测量
背景情况:
- 生成型人工智能 (AI) 显示出对文本分析的潜力,特别是在评估情感动态方面.
- 之前的研究表明,人工智能可能有助于理解文本的细微方面.
- 本研究探讨了AI在文本数据中的分析思维量化的应用.
研究的目的:
- 评估ChatGPT产生的分析思维得分与LIWC等传统的基于字典的文本分析工具之间的关联.
- 评估大型语言模型 (LLM) 在执行定量文本分析任务时的可靠性.
- 为考虑在文本分析中使用人工智能的研究人员提供经验证据.
主要方法:
- 在四个不同的样本中利用了超过16000个文本.
- 采用了三种不同的提示来查询两个大型语言模型 (GPT-3.5和GPT-4).
- 将人工智能生成的分析思维得分与已建立的语言调查和词汇计数 (LIWC) 得分进行比较.
主要成果:
- 在ChatGPT和LIWC分析思维得分之间发现了小的,统计学上显著的关联 (元分析效应大小:0.058 < rs < .304).
- 观察到,在计算LIWC分析思维指数时,ChatGPT在22%的情况中错误地执行了数学运算.
- 在定量分析中表示LLM的基本文字和数字处理中的潜在不可靠性.
结论:
- 包括ChatGPT在内的生成AI在需要精确计算的细微文本分析任务中显示出有限的可靠性.
- 研究人员应谨慎使用人工智能工具进行定量文本分析,因为可能存在不准确性.
- 需要进一步的研究来提高AI在语言研究中的数学和分析能力.
相关概念视频
01:27Language and Cognition
346
Language serves as a bridge between ideas and communication, influencing how individuals perceive and interact with the world. Psychologists have long debated whether language shapes thought or vice versa. This discussion gained grip with Edward Sapir and Benjamin Lee Whorf in the 1940s, who proposed that language determines thought, a concept known as linguistic determinism. They suggested that the vocabulary and structure of a language influence how its speakers think and perceive reality.
346
01:20Introduction to Cognitive Psychology
484
Cognitive psychology is the field of psychology dedicated to examining how people think. It attempts to explain how and why we think the way we do by studying the interactions among human thinking, emotion, creativity, language, and problem-solving, as well as other cognitive processes. Cognitive psychology studies how information is processed and manipulated in remembering, thinking, and knowing.
This field emerged in the mid-20th century, following a period dominated by behaviorism, which...
This field emerged in the mid-20th century, following a period dominated by behaviorism, which...
484
01:16Non-equilibrium in the Cell
4.4K
An important concept in studying metabolism and energy is that of chemical equilibrium. Most chemical reactions are reversible. They can proceed in both directions, releasing energy into their environment in one direction, and absorbing it from the environment in the other direction. The same is true for the chemical reactions involved in cell metabolism, such as the breaking down and building up of proteins into and from individual amino acids, respectively. Reactants within a closed system...
4.4K
01:17Cognitivism
1.4K
Cognitive psychology emerged as a significant field in the mid-20th century. It focused on understanding humans' internal mental processes. This approach emphasizes how people perceive, remember, think, and solve problems—elements critical to human cognition.
Previously dominated by behaviorism, which prioritized observable behaviors and largely ignored mental processes, psychology transformed in the 1950s. Cognitive psychologists argue that understanding how we think and process...
Previously dominated by behaviorism, which prioritized observable behaviors and largely ignored mental processes, psychology transformed in the 1950s. Cognitive psychologists argue that understanding how we think and process...
1.4K
01:37Reason and Intuition
6.5K
The human brain processes information for decision-making using one of two routes: an intuitive system and a rational system (Epstein, 1994; popularized by Kahneman, 2011 as System 1 and System 2, respectively). The intuitive system is quick, impulsive, and operates with minimal effort, relying on emotions or habits to provide cues for what to do next, while the rational system is logical, analytical, deliberate, and methodical. Research in neuropsychology suggests that the...
6.5K
01:10Higher Mental Functions of the Brain: Language
855
Language is a system of communication that allows the expression of thoughts, ideas, and feelings. The brain processes language in both hemispheres.
Language formation and comprehension take place in the dominant hemisphere. The dominant hemisphere is responsible for understanding the meaning of spoken, written, or sign language, as well as the ability to communicate. For most people, the left hemisphere is the dominant one. The right hemisphere, then, gives tone and emotional context to the...
Language formation and comprehension take place in the dominant hemisphere. The dominant hemisphere is responsible for understanding the meaning of spoken, written, or sign language, as well as the ability to communicate. For most people, the left hemisphere is the dominant one. The right hemisphere, then, gives tone and emotional context to the...
855