来自TREC简单语言生物医学摘要 (PLABA) 轨道的经验教训
Brian Ondov1, William Xia2, Kush Attal3
1Yale School of Medicine, 333 Cedar Street, New Haven, 06510, CT, USA.
ArXiv
|September 19, 2025
概括
大型语言模型 (LLM) 显示出对简化患者生物医学文本的承诺. 然而,由于潜在的危害,严格的评估至关重要,目前的自动指标需要改进.
科学领域:
- 生物医学信息学是生物医学信息学.
- 自然语言处理自然语言处理.
- 卫生沟通健康沟通
背景情况:
- 大型语言模型 (LLM) 的进步为将复杂的生物医学文献翻译成患者友好的语言提供了潜力.
- 医疗信息的高风险性质需要对LLM调整进行彻底评估,以确保患者的安全和理解.
研究的目的:
- 通过使用LLMs,刺激研究使生物医学文献适应于普通语言.
- 对系统进行高质量的评估,对生物医学摘要进行简单语言的调整.
主要方法:
- 生物医学摘要的简单语言适应 (PLABA) 在 2023-2024 文本检索会议 (TRC) 的轨道.
- 任务1:完成摘要的句子级重写. 任务2:识别和替换困难的术语.
- 使用专业编写的参考资料进行自动评估,并由生物医学专家进行广泛的手动评估.
主要成果:
- 顶级模型在重写摘要时实现了人类水平的事实准确性和完整性,但缺乏简单性和简洁性.
- 自动指标显示与抽象重写手动判断的相关性不佳.
- 系统在识别和分类困难术语方面扎,但在生成任务2的准确,完整和简单的替代品方面表现出色.
结论:
- 该PLABA轨道展示了LLMs在使生物医学文献向公众开放方面的潜力.
- 在目前的LLM系统和自动基准测试工具中发现了这个任务的缺陷.
- 需要进一步的研究来提高LLM的表现,并制定更好的评估指标,以适应普通语言.
更多相关视频
05:47Evidence-based Knowledge Synthesis and Hypothesis Validation: Navigating Biomedical Knowledge Bases via Explainable AI and Agentic Systems
Published on: June 13, 2025
1.3K
09:20Cloud-Based Phrase Mining and Analysis of User-Defined Phrase-Category Association in Biomedical Publications
Published on: February 23, 2019
9.2K
相关概念视频
02:07Improving Translational Accuracy
3.6K
3.6K
02:07Improving Translational Accuracy
14.1K
Base complementarity between the three base pairs of mRNA codon and the tRNA anticodon is not a failsafe mechanism. Inaccuracies can range from a single mismatch to no correct base pairing at all. The free energy difference between the correct and nearly correct base pairs can be as small as 3 kcal/ mol. With complementarity being the only proofreading step, the estimated error frequency would be one wrong amino acid in every 100 amino acids incorporated. However, error frequencies observed in...
14.1K
01:23Types of Biopharmaceutical Studies: Controlled and Non-Controlled Approaches
407
Biopharmaceutical studies constitute a vital field aiming to enhance drug delivery methods and refine therapeutic approaches, drawing upon diverse interdisciplinary knowledge. In research methodologies, the choice between controlled and non-controlled studies significantly influences the study's reliability and accuracy.
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast,...
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast,...
407
01:11Clinical Trials: Overview
4.6K
Clinical development focuses on how the drug will interact with the human body and encompasses four key phases of clinical trials, each serving a specific purpose in assessing the safety and effectiveness of new drugs. These phases overlap and build upon one another. Phase I involves a small group of healthy volunteers (typically 20-80 individuals) or, in cases where significant toxicity is expected, patients with the targeted disease, such as cancer or AIDS. The volunteers are tested for...
4.6K
01:28Preclinical Development: Overview
5.8K
Preclinical development consists of a series of tests that ensure the safety and efficacy of a new therapeutic compound before it is tested in humans. There are four main phases to this process. First, safety pharmacology tests are conducted to ensure the drug does not produce any acutely harmful effects. These tests examine parameters such as bronchoconstriction, cardiac dysrhythmias, blood pressure changes, and ataxia. Next, preliminary toxicological testing is performed to determine the...
5.8K
01:28Biopharmaceutics and Pharmacokinetics: Overview
3.4K
Understanding drugs, drug products, and their performance in pharmaceutical science is pivotal. Drugs, whether simple molecules or complex compounds, are designed to interact with the body's biological systems to diagnose, treat, or prevent diseases. Drug products include various delivery systems such as tablets, capsules, injections, and inhalers. The performance of these drug products is gauged by their ability to deliver the active ingredient to the desired site of action at the...
3.4K