从临床报告中提取人类水平的信息,使用微调的语言模型
Longchao Liu1, Long Lian1, Yiyan Hao2
1Electrical Engineering and Computer Sciences, UC Berkeley, 387 Soda Hall, Berkeley, CA, 94720, USA.
Scientific reports
|November 25, 2025
概括
开源的大型语言模型 (LLM) 可以从临床笔记中提取结构化数据,使用最小的资源,以人类级准确度. 斯特拉塔图书馆和精心调整的LLM,如Llama-3.1,证明了高效,可访问的临床研究数据库的创建.
科学领域:
- 生物医学信息学 生物医学信息学
- 自然语言处理自然语言处理.
- 临床研究数据管理数据管理
背景情况:
- 从非结构化的临床笔记中提取结构化数据是临床研究中的一个重大挑战.
- 现有的方法往往需要大量的计算和注释资源.
- 需要为临床数据提取提供可访问,高效的工具.
研究的目的:
- 评估开源大型语言模型 (LLM) 的有效性,以最小的资源从临床报告创建高质量的研究数据库.
- 介绍Strata,一个低代码库,旨在促进基于LLM的从临床笔记中提取数据.
- 将各种开源LLM与GPT-4和人类注释器的性能进行比较.
主要方法:
- 开发Strata,一个用于LLM驱动数据提取的低代码库.
- 由训练有素的研究人员对四个不同的临床数据集 (前列腺MRI,乳腺病理学,脏病理学,MDS病理学) 进行注释.
- 使用Strata.使用多个开源LLM (指令调整,药物特定,基于推理,LoRA精细调整) 的评估.
- 对LLM性能与零射击GPT-4和基于精确匹配精度的第二个人类注释器进行比较.
主要成果:
- 精细调整的LoRa Llama-3.1 8B实现了与人类注释器不逊色的性能,平均准确匹配准确率为90.0%.
- 微调的Llama-3.1显著优于其他开源模型,包括DeepSeekR1-Distill-Llama (56.8%准确率) 和Llama-3-8B-UltraMedical (39.1%准确率).
- 在大多数数据集中,GPT-4表现出非劣质的性能,而小型的开源LLM则使用有限的培训数据和桌面硬件实现了人类水平的准确性.
结论:
- 小型的,开源的LLM,当与Strata一起利用时,为策划本地研究数据库提供了可访问和有效的解决方案.
- 这些模型为临床数据提取提供了人类水平的准确性,使用最小的计算资源 (桌面级硬件,<100个培训报告).
- 开源的LLM允许本地托管和版本控制,在临床研究数据管理的商业替代方案上提供优势.
更多相关视频
03:14Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
Published on: December 6, 2024
1000
05:47Evidence-based Knowledge Synthesis and Hypothesis Validation: Navigating Biomedical Knowledge Bases via Explainable AI and Agentic Systems
Published on: June 13, 2025
1.3K
相关概念视频
01:11Clinical Trials: Overview
4.5K
Clinical development focuses on how the drug will interact with the human body and encompasses four key phases of clinical trials, each serving a specific purpose in assessing the safety and effectiveness of new drugs. These phases overlap and build upon one another. Phase I involves a small group of healthy volunteers (typically 20-80 individuals) or, in cases where significant toxicity is expected, patients with the targeted disease, such as cancer or AIDS. The volunteers are tested for...
4.5K
02:07Improving Translational Accuracy
3.5K
3.5K
01:16Clinical Trials
10.1K
Clinical trials are prospective experimental studies conducted on humans to determine the safety and efficacy of treatments, drugs, diet methods, and medical devices. Using statistics in clinical trials enables researchers to derive reasonable and accurate conclusions from the collected data, allowing them to make wise decisions in uncertain situations. In medical research, statistical methods are crucial for preventing errors and bias.
There are four phases in a clinical trial. A phase one...
There are four phases in a clinical trial. A phase one...
10.1K
01:26Types of Reports III: Telephone and Verbal Reports
972
Telephone and Verbal Reports in healthcare settings are two communication methods for conveying therapeutic instructions from healthcare providers to nurses or other healthcare staff.
Here's an overview of each type:
Telephone Orders
Here's an overview of each type:
Telephone Orders
972