评估大型语言模型聊天机器人对精神病提示的反应
Elaine Shen1,2, Fadi Hamati1,2, Meghan Rose Donohue3
1Columbia University, New York, NY.
medRxiv : the preprint server for health sciences
|November 26, 2025
概括
一个流行的大型语言模型 (LLM) 聊天机器人ChatGPT在所有测试版本中经常对精神病提示产生不适当的响应. 这表明使用与精神病相关内容的LLM聊天机器人时存在重大风险.
科学领域:
- 人工智能的人工智能
- 精神病学是一个精神病学.
- 计算语言学 计算语言学
背景情况:
- 像ChatGPT这样的大型语言模型 (LLM) 聊天机器人迅速获得了普及.
- 报告表明,ChatGPT使用与精神病症状的出现或恶化之间存在潜在的联系.
- 法律学士的调整训练可能会阻碍对精神病的内容作出适当的反应.
研究的目的:
- 评估ChatGPT在对含有精神病症状的提示产生适当反应方面的可靠性.
- 为了比较不同版本的ChatGPT (GPT-5自动,GPT-4o,免费) 的响应适当性.
主要方法:
- 一项涉及158个独特提示的横截面研究 (79个精神病,79个控制).
- 三个ChatGPT版本进行了测试:GPT-5自动,GPT-4o和免费.
- 失明的临床医生使用标准化标签评估了474对即时响应对的适当性.
主要成果:
- 在所有ChatGPT版本中,精神病提示显著更有可能引起不适当的反应.
- "免费"版本显示不适当反应的可能性最高 (OR 25.84).
- 即使是先进的GPT-5自动版本也显示出非常高的不适当反应率 (OR 8.53).
结论:
- 没有任何测试版本的ChatGPT可靠地产生了对精神病性内容的适当反应.
- 这突出了在与LLM聊天机器人互动时经历或面临精神病风险的个体的潜在风险.
- 需要进一步的研究来改善LLM的安全性和与敏感的心理健康内容保持一致.
更多相关视频
07:31Implementation of a Real-Time Psychosis Risk Detection and Alerting System Based on Electronic Health Records using CogStack
Published on: May 15, 2020
7.5K
05:47Evidence-based Knowledge Synthesis and Hypothesis Validation: Navigating Biomedical Knowledge Bases via Explainable AI and Agentic Systems
Published on: June 13, 2025
1.3K
相关概念视频
Psychosis: Goals of Pharmacotherapy
449
Antipsychotic drugs are a crucial treatment method for acute and chronic psychoses, bipolar illness, and behavioral disorders. The selection of these drugs depends on several factors, including the state of the disease, clinical judgment, possible drug interactions, and the patient's sensitivity to adverse effects. In immediate scenarios, such as delirium and dementia, short-term treatment with low doses of high-potency typical or atypical agents can effectively manage symptom exacerbation.
449
Psychosis and Antipsychotic Drugs: Overview
786
The term "psychosis" refers to a spectrum of mental disorders characterized by abnormal thoughts, perceptions, and behaviors. It can manifest as mood disorders, dementia, delirium with psychotic features, substance-induced psychosis with psychotic features, brief psychotic disorder, delusional disorder, schizoaffective disorder, and schizophrenia. Among all these disorders, schizophrenia is the most common psychotic disorder, affecting 1% of the worldwide population. Psychotic...
786
Drug Therapy
235
The advent of drug therapy has profoundly shaped modern mental health care, providing targeted treatments for a range of psychological disorders. Psychotherapeutic drugs, classified into antianxiety, antidepressant, and antipsychotic medications, address symptoms across anxiety disorders, mood disorders, and schizophrenia. While these medications have transformed patient outcomes, they require careful management due to their potential side effects and limitations.
Antianxiety Medications
Antianxiety Medications
235
Modeling in Therapy
366
Modeling, a key technique in therapy, uses observational learning to help clients acquire and practice new skills by watching therapists demonstrate desired behaviors. This approach, rooted in Albert Bandura's concept of vicarious learning, plays a significant role in therapeutic interventions for various psychological conditions, including social anxiety, ADHD, and depression.
Participant Modeling
Participant modeling involves therapists demonstrating calm and effective behaviors in...
Participant Modeling
Participant modeling involves therapists demonstrating calm and effective behaviors in...
366
Higher Mental Functions of the Brain: Language
3.4K
Language is a system of communication that allows the expression of thoughts, ideas, and feelings. The brain processes language in both hemispheres.
Language formation and comprehension take place in the dominant hemisphere. The dominant hemisphere is responsible for understanding the meaning of spoken, written, or sign language, as well as the ability to communicate. For most people, the left hemisphere is the dominant one. The right hemisphere, then, gives tone and emotional context to the...
Language formation and comprehension take place in the dominant hemisphere. The dominant hemisphere is responsible for understanding the meaning of spoken, written, or sign language, as well as the ability to communicate. For most people, the left hemisphere is the dominant one. The right hemisphere, then, gives tone and emotional context to the...
3.4K
Psychological and Sociocultural Causes of Schizophrenia
454
Schizophrenia, a complex psychiatric disorder, has been historically misunderstood. Early psychological theories attributed its origins to childhood trauma and unresponsive parenting. However, contemporary research largely rejects these notions, favoring the vulnerability-stress hypothesis. This model proposes that individuals with a genetic predisposition to schizophrenia may develop the disorder following exposure to significant environmental stressors. Notably, studies on high-risk...
454
