利用开源的大型语言模型在资源有限的环境中提取临床信息
Luc Builtjes1, Joeran Bosma1, Mathias Prokop1
1Department of Radiology and Nuclear Medicine, Radboud University Medical Center, 6525GA Nijmegen, The Netherlands.
JAMIA open
|October 3, 2025
概括
开源大型语言模型 (LLM) 在从荷兰临床文本中提取信息时显示出强大的零射击性能. 像Llama-3.3-70B这样的模型非常出色,特别是在回归任务中,为NLP挑战提供了可行的替代方案.
科学领域:
- 自然语言处理 (NLP) 是一种自然语言处理.
- 人工智能 (AI) 是一种人工智能.
- 临床信息学 临床信息学
背景情况:
- 从荷兰医疗报告中提取临床信息是具有挑战性的,因为语言障碍和资源有限.
- 开源生成型大语言模型 (LLM) 为自动化文本分析提供了潜在的解决方案.
研究的目的:
- 评估开源生成LLM在从荷兰医疗报告中提取临床信息上的零射击性能.
- 通过使用DRAGON基准来评估LLM在分类,回归和命名实体识别 (NER) 任务中的能力.
主要方法:
- 开发了llm_extractinator框架,用于可扩展的,开源的临床文本信息提取.
- 在零射击环境中对28个DRAGON基准任务进行了9个多语言LLM的评估.
- 研究了英语语文翻译对模型性能的影响.
主要成果:
- 拉玛-3.3-70B获得了最高的实用性得分 (0.760),其他模型如Phi-4-14B和Qwen-2.5-14B也表现良好.
- 在17个任务中,LLM的表现超过或与微调的罗伯塔基线相匹配,特别是在回归和结构化分类方面.
- 在所有模型中,命名实体识别 (NER) 性能始终较低,而向英语翻译通常会降低性能.
结论:
- 开源生成的LLM在临床NLP任务中表现出显著的零射击能力,特别是在结构化推理中.
- 围绕14B参数的模型提供了性能和计算成本的良好平衡,尽管较大的模型领先.
- 母语支持至关重要,因为向英语翻译会对表现产生负面影响,这凸显了低资源和多语言环境的价值.
更多相关视频
05:47Evidence-based Knowledge Synthesis and Hypothesis Validation: Navigating Biomedical Knowledge Bases via Explainable AI and Agentic Systems
Published on: June 13, 2025
1.3K
07:50A Metadata Extraction Approach for Clinical Case Reports to Enable Advanced Understanding of Biomedical Concepts
Published on: September 20, 2018
16.4K
相关概念视频
02:07Improving Translational Accuracy
14.1K
Base complementarity between the three base pairs of mRNA codon and the tRNA anticodon is not a failsafe mechanism. Inaccuracies can range from a single mismatch to no correct base pairing at all. The free energy difference between the correct and nearly correct base pairs can be as small as 3 kcal/ mol. With complementarity being the only proofreading step, the estimated error frequency would be one wrong amino acid in every 100 amino acids incorporated. However, error frequencies observed in...
14.1K
02:07Improving Translational Accuracy
3.5K
3.5K
00:56Extraction: Advanced Methods
1.1K
Metal ions can be separated from one another by complexation with organic ligands–the chelating agent– to form uncharged chelates. Here, the chelating agent must contain hydrophobic groups and behave as a weak acid, losing a proton to bind with the metal. Since most organic ligands used in this process are insoluble or undergo oxidation in the aqueous phase, the chelating agent is initially added to the organic phase and extracted into the aqueous phase. The metal-ligand complex is...
1.1K
01:27Nursing Clinical Information System
1.2K
Nursing Clinical Information System (NCIS)
A Nursing Clinical Information System (NCIS) is a specialized type of healthcare information system tailored to meet the unique needs of nursing practice. It incorporates the principles of nursing informatics to streamline information management and improve the quality of care delivery.
Critical attributes of NCIS include:
A Nursing Clinical Information System (NCIS) is a specialized type of healthcare information system tailored to meet the unique needs of nursing practice. It incorporates the principles of nursing informatics to streamline information management and improve the quality of care delivery.
Critical attributes of NCIS include:
1.2K
01:11Clinical Trials: Overview
4.6K
Clinical development focuses on how the drug will interact with the human body and encompasses four key phases of clinical trials, each serving a specific purpose in assessing the safety and effectiveness of new drugs. These phases overlap and build upon one another. Phase I involves a small group of healthy volunteers (typically 20-80 individuals) or, in cases where significant toxicity is expected, patients with the targeted disease, such as cancer or AIDS. The volunteers are tested for...
4.6K