主题熟悉性和即时具体性对引用制造在心理健康研究中使用大语言模型的影响:实验研究实验研究
Jake Linardon1, Hannah K Jarman1, Zoe McClure1
1School of Psychology, Faculty of Health, Deakin University, Geelong, Victoria, Australia.
JMIR mental health
|November 12, 2025
概括
像GPT-4o这样的大型语言模型 (LLM) 经常在心理健康评论中捏造引用,近三分之二的引用是不准确的. 可靠性因主题可见性和及时特异性而异,强调需要人类验证.
科学领域:
- 人工智能在心理健康研究中的应用
- 自然语言处理应用程序
- 学术沟通诚信学术沟通诚信
背景情况:
- 大型语言模型 (LLM) 越来越多地用于心理健康研究,以提高效率.
- 法律法规可以产生可信但伪造的内容,包括不存在的文献引用.
- 关于引用伪造的先前研究缺乏对不同公共知名度和科学成熟度的心理健康主题进行系统分析.
研究的目的:
- 为了评估 GPT-4o 中的引用伪造和文献错误,生成了关于心理健康主题的文献评论.
- 调查心理健康障碍的公众熟悉度和科学成熟度如何影响引用准确度.
- 为了确定提示特异性 (一般与专业) 是否会影响制造和准确率.
主要方法:
- GPT-4o产生了六篇文学评论 (~2000字,≥20个引文) 关于主要抑郁症 (高可见性),暴饮暴食障碍 (中度) 和身体形障碍 (低).
- 在一般 (症状,影响,治疗) 和专业 (数字干预) 层面进行了审查.
- 总共有176个引文被提取并使用多个学术数据库进行验证; 伪造率和准确率使用千平方测试进行了比较.
主要成果:
- GPT-4o产生了176个引用,其中35个 (19.9%) 是伪造的,64个 (45.4%) 的真实引用包含错误.
- 根据疾病的制造率有显著差异 (P=.001),与主要抑郁障碍 (6%) 相比,暴饮暴食障碍 (28%) 和身体形障碍 (29%) 的制造率更高.
- 专业审查显示,暴饮暴食障碍的伪造率更高 (46%对17%;P=.01),准确性因疾病和审查类型而异.
结论:
- 引用伪造和文献错误在GPT-4o输出中很普遍,影响了近三分之二的引用.
- 法学士生成引文的可靠性受到障碍熟悉性和快速特异性的影响,对不太明显或专业的主题构成更大的风险.
- 严格的人员验证,谨慎的快速设计和机构保障对于保持LLM整合的研究完整性至关重要.
更多相关视频
相关概念视频
Higher Mental Functions of the Brain: Language
3.4K
Language is a system of communication that allows the expression of thoughts, ideas, and feelings. The brain processes language in both hemispheres.
Language formation and comprehension take place in the dominant hemisphere. The dominant hemisphere is responsible for understanding the meaning of spoken, written, or sign language, as well as the ability to communicate. For most people, the left hemisphere is the dominant one. The right hemisphere, then, gives tone and emotional context to the...
Language formation and comprehension take place in the dominant hemisphere. The dominant hemisphere is responsible for understanding the meaning of spoken, written, or sign language, as well as the ability to communicate. For most people, the left hemisphere is the dominant one. The right hemisphere, then, gives tone and emotional context to the...
3.4K
Stereotype Content Model
15.3K
The Stereotype Content Model (SCM) was first proposed by Susan Fiske and her colleagues (Fiske, Cuddy, Glick & Xu, 2002; see also Fiske, 2012 and Fiske, 2017). The SCM specifies that when someone encounters a new group, they will stereotype them based on two metrics: warmth—or that group’s perceived intent, and how likely they are to provide help or inflict harm—and competence—or their ability to carry out that objective. Depending on the warmth-competence...
15.3K
Fundamental Attribution Error
13.7K
According to some social psychologists, people tend to overemphasize internal factors as explanations—or attributions—for the behavior of other people. They tend to assume that the behavior of another person is a trait of that person, and to underestimate the power of the situation on the behavior of others. They tend to fail to recognize when the behavior of another is due to situational variables, and thus to the person’s state. This erroneous assumption is...
13.7K
Hindsight Biases
4.2K
Hindsight bias leads you to believe that the event you just experienced was predictable, even though it really wasn’t. In other words, you knew all along that things would turn out the way they did. Can you relate this to the phrase "Hindsight is 20/20" now?
4.2K
Stereotype Threat and Self-fulfilling Prophecies
41.9K
When we hold a stereotype about a person, we have expectations that he or she will fulfill that stereotype. A self-fulfilling prophecy is an expectation held by a person that alters his or her behavior in a way that tends to make it true. When we hold stereotypes about a person, we tend to treat the person according to our expectations. This treatment can influence the person to act according to our stereotypic expectations, thus confirming our stereotypic beliefs. Research by Rosenthal and...
41.9K
The Anchoring-and-Adjustment Heuristic
7.7K
In order to make good decisions, we use our knowledge and our reasoning. Often, this knowledge and reasoning is sound and solid. However, sometimes, we are swayed by biases or by others manipulating a situation. For example, let’s say you and three friends wanted to rent a house and had a combined target budget of $1,600. The realtor shows you only very run-down houses for $1,600 and then shows you a very nice house for $2,000. Might you ask each person to pay more in rent to get the...
7.7K


