协作大型语言模型用于在生活系统审查中自动提取数据
Muhammad Ali Khan1, Umair Ayub1, Syed Arsalan Ahmed Naqvi1
1Department of Medicine, Mayo Clinic, Phoenix, United States of America.
medRxiv : the preprint server for health sciences
|October 14, 2024
概括
在两位审核员的模拟中,使用大型语言模型 (LLM) 进行自动数据提取显示了活系统性审核 (LSR) 的前景. 不一致的LLM反应的交叉批评提高了准确性,支持有效的证据综合.
科学领域:
- 医疗信息学 医疗信息学
- 医疗保健中的人工智能
- 系统审查方法论 系统审查方法论
背景情况:
- 数据提取是进行生活系统审查 (LSRs) 的重要瓶.
- 目前的方法是劳动密集型的,阻碍了及时合成证据.
- 需要有效的,自动化的数据提取方法.
研究的目的:
- 使用大语言模型 (LLM) 开发和评估一个可泛化的,自动化的数据提取工作流.
- 模拟两个审核员的数据提取过程,以提高准确性和可靠性.
- 评估LLM在提取LSR数据方面的表现.
主要方法:
- 使用了已发表的LSR数据集,包括10个临床试验和22个出版物.
- 两个LLM,GPT-4-turbo和Claude-3-Opus,被用来进行数据提取.
- 对不一致的LLM响应实施了交叉批评机制,随后对黄金标准进行了准确性评估.
主要成果:
- 在快速开发组中,96%的LLM答案与0.99准确度一致.
- 在持有测试组中,87%的答案与0.94准确度一致.
- 交叉批评解决了51%的不一致反应,将其准确度提高到0.76.
结论:
- 在模拟的两审核员工作流程中,LLM驱动的数据提取显示了LSRs的合理性能.
- 一致的LLM响应通常是准确的,交叉批评增强了不一致的答案的准确性.
- 这种自动化方法有助于创建真正"活的"系统审查.
相关概念视频
01:17Systematic Sampling Method
10.3K
Sampling is a technique to select a portion (or subset) of the larger population and study that portion (the sample) to gain information about the population. Data are the result of sampling from a population. The sampling method ensures that samples are drawn without bias and accurately represent the population. Because measuring the entire population in a study is not practical, researchers use samples to represent the population of interest.
Systematic sampling is one of the simplest methods...
Systematic sampling is one of the simplest methods...
10.3K
00:56Extraction: Advanced Methods
1.2K
Metal ions can be separated from one another by complexation with organic ligands–the chelating agent– to form uncharged chelates. Here, the chelating agent must contain hydrophobic groups and behave as a weak acid, losing a proton to bind with the metal. Since most organic ligands used in this process are insoluble or undergo oxidation in the aqueous phase, the chelating agent is initially added to the organic phase and extracted into the aqueous phase. The metal-ligand complex is...
1.2K