医疗大型语言模型容易受到数据中毒攻击的攻击
Daniel Alexander Alber1,2, Zihao Yang3,4, Anton Alyakin3,5
1Department of Neurosurgery, NYU Langone Health, New York, NY, USA. daniel.alber@nyulangone.org.
Nature medicine
|January 8, 2025
概括
医疗保健中的大型语言模型 (LLM) 存在传播医学错误信息的风险. 一小部分受毒的数据会产生有害的模型,标准测试无法检测到,但知识图的方法可以减轻风险.
科学领域:
- 人工智能的人工智能
- 生物医学信息学 生物医学信息学
- 数据科学数据科学数据科学
背景情况:
- 大型语言模型 (LLM) 在医疗保健中越来越多地被采用.
- 在互联网上接受培训的LLM可能会吸收和传播医学错误信息.
- 现有的基准标准可能无法从中毒的培训数据中检测到微妙的危害.
研究的目的:
- 评估受过毒数据培训的LLM传播医学错误信息的风险.
- 评估标准基准在检测数据中毒导致的LLM损害方面的有效性.
- 通过生物医学知识图表提出和验证一个减轻危害的策略.
主要方法:
- 模拟了对"The Pile"数据集的数据中毒攻击,这是一个常见的LLM培训资源.
- 评估了与标准开源基准相对应的损坏的LLM性能.
- 开发并测试了使用生物医学知识图表来选LLM输出的危害减轻算法.
主要成果:
- 仅用错误信息取代0.001%的培训令牌,就创造了有害的LLMs.
- 腐败的LLM在标准基准上表现相当于不受腐败的LLM.
- 提出的基于知识图的策略捕获了91.9%的有害内容 (F1 = 85.7%).
结论:
- 在网络取数据上接受培训的LLM在医疗保健中存在医疗错误信息的重大风险.
- 标准的LLM评估基准不足以检测数据中毒引起的医疗危害.
- 生物医学知识图为验证LLM输出和减轻安全风险提供了一个有希望的方法.
相关概念视频
Anticholinesterase Agents: Poisoning and Treatment
783
Anticholinesterases, also known as cholinesterase inhibitors, work by blocking the breakdown of acetylcholine, leading to its accumulation in the synaptic cleft. This accumulation indirectly enhances both muscarinic and nicotinic actions. These agents are classified as reversible or irreversible based on their mechanism of action.
Irreversible agents form a strong bond with the cholinesterase enzyme, making it inactive. The breakdown of the phosphorylated enzyme is...
Irreversible agents form a strong bond with the cholinesterase enzyme, making it inactive. The breakdown of the phosphorylated enzyme is...
783
Prevention of Further Absorption of Poison
779
In cases of acute poisoning, the primary objective is to prevent further absorption of the toxic substance into the body. Immediate interventions using various decontamination techniques targeting the gastrointestinal (GI) tract can achieve this. Decontamination is crucial to prevent poison from entering the systemic circulation, which involves washing affected areas with water and mild soap and removing contaminated clothing. Once external decontamination is done, attention must be turned to...
779
Pharmacokinetic Models: Overview
577
Pharmacokinetic models utilize mathematical analysis to achieve a detailed quantitative understanding of a drug's life cycle within the body. They are instrumental in simulating a drug's pharmacokinetic parameters, predicting drug concentrations over time, optimizing dosage regimens, linking concentrations with pharmacologic activity, and estimating potential toxicity.
There are three primary types of models: empirical, compartment, and physiological. Empirical models, with minimal...
There are three primary types of models: empirical, compartment, and physiological. Empirical models, with minimal...
577
Pharmacovigilance
771
Post-marketing surveillance is a critical component of pharmaceutical regulation, often uncovering unanticipated adverse drug reactions (ADRs) once a drug is widely used over an extended period.
This process, termed pharmacovigilance, aims to detect, evaluate, and minimize harmful effects related to medication use. The data collection for pharmacovigilance depends on spontaneous reporting systems, where healthcare professionals or patients voluntarily report suspected ADRs.
In some cases, there...
This process, termed pharmacovigilance, aims to detect, evaluate, and minimize harmful effects related to medication use. The data collection for pharmacovigilance depends on spontaneous reporting systems, where healthcare professionals or patients voluntarily report suspected ADRs.
In some cases, there...
771
Factors Affecting the Risk of Infection
11.3K
The hosts' susceptibility to infection depends on several factors. The integrity of the skin and mucous membranes helps protect the body against microbial attacks. When the skin is altered, the chance of infection, limb loss, and even death increases.
The integrity and count of the white blood cells help the body resist pathogens and fight infection. When impaired, it reduces the body's resistance to pathogens. The acidic pH levels of the gastrointestinal, genitourinary tracts, and skin...
The integrity and count of the white blood cells help the body resist pathogens and fight infection. When impaired, it reduces the body's resistance to pathogens. The acidic pH levels of the gastrointestinal, genitourinary tracts, and skin...
11.3K
Leaky Scanning
5.1K
During most eukaryotic translation processes, the small 40S ribosome subunit scans an mRNA from its 5' end until it encounters the first start AUG codon. The large 60S ribosomal subunit then joins the smaller one to initiate protein synthesis. The location of the translation initiation is largely determined by the nucleotides near the start codon as there may be multiple translation initiation sites present on the mRNA. Marilyn Kozak discovered that the sequence RCCAUGG (where R...
5.1K


