医学界的红色团队ChatGPT将对模型行为产生真实世界的见解
Crystal T Chang1,2, Hodan Farah1, Haiwen Gui1,3
1Department of Dermatology, Stanford University, Stanford, USA.
NPJ digital medicine
|March 7, 2025
概括
医疗保健大语言模型 (LLM) 需要严格的红色团队,以确保安全性和准确性. 我们的研究发现了大量不适当的反应,突出了在临床环境中对AI的持续评估的需要.
科学领域:
- 医疗信息学 医疗信息学
- 医疗保健中的人工智能
- 临床决策支持系统 临床决策支持系统
背景情况:
- 在大型语言模型 (LLM) 中,红色团队对于识别非故意行为至关重要.
- 尽管AI具有至关重要的重要性,但在医疗保健领域的独立红色团队工作是有限的.
- 确保临床环境中LLMs的安全性,隐私性,准确性和公平性至关重要.
研究的目的:
- 通过对抗性测试建立一个基准来评估大型语言模型 (LLM) 在医疗保健中的性能.
- 评估LLM对现实世界临床场景的反应是否适当.
- 识别和分类不适当的反应类型,包括安全,隐私,准确性和偏见问题.
主要方法:
- 召集了一组多元化的80名参与者 (临床医生,学生,技术专业人员) 进行红色团队合作.
- 开发了376个基于真实世界临床病例的独特提示,以压力测试LLMs.
- 在安全,隐私,幻觉/准确性和偏见方面分类不适当的反应.
- 六名经过医学培训的审稿人分析并注释了即时响应对.
主要成果:
- 在所有测试模型中,1504个总答案中,有20.1%被认为是不合适的.
- GPT-3.5显示了25.8%的不适当响应率,而GPT-4.0显示了16% (没有互联网) 和17.8% (有互联网).
- 值得注意的是,对于GPT-3.5来说适当的响应中有21.5%在更新的模型中变得不合适,这表明性能不断变化和潜在的回归.
结论:
- 独立的红色团队对于推动在医疗保健中安全和公平地部署LLM至关重要.
- 开发的基准为临床AI工具的代评估和改进提供了一个框架.
- 持续评估是必要的,以解决LLM绩效的动态性质,并确保临床效用.
相关概念视频
Modeling in Therapy
39
Modeling, a key technique in therapy, uses observational learning to help clients acquire and practice new skills by watching therapists demonstrate desired behaviors. This approach, rooted in Albert Bandura's concept of vicarious learning, plays a significant role in therapeutic interventions for various psychological conditions, including social anxiety, ADHD, and depression.
Participant Modeling
Participant modeling involves therapists demonstrating calm and effective behaviors in...
Participant Modeling
Participant modeling involves therapists demonstrating calm and effective behaviors in...
39
Regression Toward the Mean
6.3K
Regression toward the mean (“RTM”) is a phenomenon in which extremely high or low values—for example, and individual’s blood pressure at a particular moment—appear closer to a group’s average upon remeasuring. Although this statistical peculiarity is the result of random error and chance, it has been problematic across various medical, scientific, financial and psychological applications. In particular, RTM, if not taken into account, can interfere when...
6.3K
Robbers Cave
14.2K
During the 1950s, the landmark Robbers Cave experiment demonstrated that when groups must compete with one another, intergroup conflict, hostility, and even violence may result. At the Oklahoman summer camp, two troops of boys—termed the Rattlers and the Eagles—took part in a week-long tournament. During this time, their negativity culminated in derogatory name-calling, fistfights, and even vandalism and destruction of property. However, this work also revealed that such tension...
14.2K
Methods of Documentation VI: Case Management Model
557
The case management model is a multidisciplinary approach that involves healthcare professionals from diverse disciplines, such as physicians, nurses, therapists, social workers, and pharmacists, working collaboratively to address the various needs of patients. Each healthcare professional brings unique expertise and perspectives, contributing to a more comprehensive understanding of the patient's condition and tailoring treatment plans accordingly.
For example, a patient with a chronic...
For example, a patient with a chronic...
557
Group Therapy
21
Group therapy is a sociocultural approach to psychological treatment, where individuals with shared psychological challenges come together under the guidance of a mental health professional. This therapeutic modality offers unique opportunities for individuals to connect, share, and grow within the context of a supportive group. By fostering mutual understanding and collaboration, group therapy can address a range of psychological concerns effectively, often complementing or surpassing the...
21
Groupthink
44.1K
When in group settings, we are often influenced by the thoughts, feelings, and behaviors around us. Groupthink is another phenomenon of conformity where modification of the opinions of members in a group aligns with what they believe is the group consensus (Janis, 1972). In such situations, the group often takes action that individuals would not perform outside the group setting because groups make more extreme decisions than individuals do. Moreover, groupthink can hinder opposing trains of...
44.1K


