协作大型语言模型用于在活系统性审查中自动提取数据.
Muhammad Ali Khan1, Umair Ayub1, Syed Arsalan Ahmed Naqvi1
1Department of Medicine, Mayo Clinic, Phoenix, AZ, 85054, United States.
Journal of the American Medical Informatics Association : JAMIA
|January 21, 2025
概括
使用大型语言模型 (LLM) 在模拟的两位审稿人的过程中自动提取数据,显示了活系统性审稿的前景. 对不一致的反应进行交叉批评可以显著提高准确性.
科学领域:
- 医疗信息学 医疗信息学
- 医疗保健中的人工智能
- 系统审查方法论 系统审查方法论
背景情况:
- 数据提取是进行生活系统审查 (LSRs) 的耗时瓶.
- 自动化这一过程对于高效和最新的证据综合至关重要.
研究的目的:
- 开发和评估使用大型语言模型 (LLM) 的可通用,自动化数据提取工作流.
- 模仿现实世界的两位审核员流程,以提高LSR数据提取的准确性.
主要方法:
- 利用来自已发表的LSR的22个出版物的数据集,专注于23个关键变量.
- 采用GPT-4-turbo和Claude-3-Opus进行数据提取,模拟两个审核员的工作流程,对不一致的响应进行交叉批评.
- 使用准确度指标与黄金标准对比评估绩效.
主要成果:
- 在快速开发组中观察到高一致性 (96%) 和准确性 (0.99).
- 在持有测试组中,87%的答案与0.94准确度一致.
- 交叉批评解决了51%的不一致反应,将其准确度提高到0.76.
结论:
- 一致的LLM响应通常是准确的,交叉批评有效地提高了不一致的提取的准确性.
- 基于LLM的模拟双审核员工作流提供了一种有效的数据提取方法,使真正活跃的系统审核成为可能.
相关概念视频
01:17Systematic Sampling Method
10.3K
Sampling is a technique to select a portion (or subset) of the larger population and study that portion (the sample) to gain information about the population. Data are the result of sampling from a population. The sampling method ensures that samples are drawn without bias and accurately represent the population. Because measuring the entire population in a study is not practical, researchers use samples to represent the population of interest.
Systematic sampling is one of the simplest methods...
Systematic sampling is one of the simplest methods...
10.3K
00:56Extraction: Advanced Methods
1.2K
Metal ions can be separated from one another by complexation with organic ligands–the chelating agent– to form uncharged chelates. Here, the chelating agent must contain hydrophobic groups and behave as a weak acid, losing a proton to bind with the metal. Since most organic ligands used in this process are insoluble or undergo oxidation in the aqueous phase, the chelating agent is initially added to the organic phase and extracted into the aqueous phase. The metal-ligand complex is...
1.2K