医学大语言模型容易受到有针对性的错误信息攻击
Tianyu Han1, Sven Nebelung2, Firas Khader2
1Department of Diagnostic and Interventional Radiology, University Hospital Aachen, Aachen, Germany. than@ukaachen.de.
NPJ digital medicine
|October 24, 2024
概括
大型语言模型 (LLM) 可以被操纵以注入不正确的医学事实,从而损害其可靠性. 这一漏洞凸显了医疗保健人工智能应用中需要加强安全措施的必要性.
科学领域:
- 人工智能在医学中的应用
- 生物医学信息学 生物医学信息学
- 机器学习安全 机器学习安全
背景情况:
- 大型语言模型 (LLM) 拥有广泛的医学知识和推理能力.
- 在医疗保健中的各种应用中,LLM显示出显著的前景.
- 确保医学LLM的可靠性和安全性至关重要.
研究的目的:
- 调查医学领域内大型语言模型中的特定漏洞.
- 证明故意注入不正确的生物医学信息的可能性.
- 评估这种操纵对模型性能和可靠性的影响.
主要方法:
- 针对LLM权重的一小部分 (1.1%) 的有针对性的操纵.
- 蓄意注入错误的生物医学事实.
- 在一个包含1025个错误事实的数据集中验证发现.
- 在操纵后对其他生物医学任务的绩效评估.
主要成果:
- 成功地将错误的生物医学事实注入到LLMs.
- 在模型输出中传播错误信息.
- 在未受影响的生物医学任务上保持性能.
- 对有针对性的操纵表现出一种特殊的易感性.
结论:
- 在医学领域,LLM对事实操纵表现出令人担忧的脆弱性.
- 这种脆弱性对医疗保健应用程序构成重大安全和可信度风险.
- 强有力的保护措施,验证和访问控制对于安全的医学LLM部署至关重要.
相关概念视频
Leaky Scanning
5.1K
During most eukaryotic translation processes, the small 40S ribosome subunit scans an mRNA from its 5' end until it encounters the first start AUG codon. The large 60S ribosomal subunit then joins the smaller one to initiate protein synthesis. The location of the translation initiation is largely determined by the nucleotides near the start codon as there may be multiple translation initiation sites present on the mRNA. Marilyn Kozak discovered that the sequence RCCAUGG (where R...
5.1K
Microorganisms in Medicine and Therapeutics
1
Microorganisms play a fundamental role in vaccine development, gene therapy, and therapeutic production. Their biological properties are harnessed to advance medicine and public health. Beyond immunization, microorganisms contribute to gut health, antibiotic synthesis, and genetic disease treatment.Live Attenuated and Inactivated VaccinesLive attenuated vaccines, such as the measles, mumps, and rubella (MMR) vaccine, utilize weakened forms of pathogens to closely resemble natural infections.
1
Mismatch Repair
4.8K
Organisms are capable of detecting and fixing nucleotide mismatches that occur during DNA replication. This sophisticated process requires identifying the new strand and replacing the erroneous bases with correct nucleotides. Mismatch repair is coordinated by many proteins in both prokaryotes and eukaryotes.
The Mutator Protein Family Plays a Key Role in DNA Mismatch Repair
The human genome has more than 3 billion base pairs of DNA per cell. Prior to cell division, that vast amount of genetic...
The Mutator Protein Family Plays a Key Role in DNA Mismatch Repair
The human genome has more than 3 billion base pairs of DNA per cell. Prior to cell division, that vast amount of genetic...
4.8K
MicroRNAs
3.0K
MicroRNA (miRNA) are short, regulatory RNA transcribed from introns (non-coding regions of a gene) or intergenic regions (stretches of DNA present between genes). Several processing steps are required to form biologically active, mature miRNA. The initial transcript, called primary miRNA (pri-mRNA), base-pairs with itself, forming a stem-loop structure. Within the nucleus, an endonuclease enzyme, called Drosha, shortens the stem-loop structure into hairpin-shaped pre-miRNA. After the pre-miRNA...
3.0K
Improving Translational Accuracy
2.5K
2.5K
Health Literacy
4.0K
Health literacy is an individual's or a community's capacity to comprehend, receive, read, and use relevant healthcare information and services. The World Health Organization (WHO, 2018) defines health literacy as the cognitive and social skills that determine the ability of individuals to gain access to, understand, and use information in ways that promote and maintain good health. As a result, the WHO helps individuals manage long-term health concerns, participate in preventative...
4.0K


