使用人工智能工具作为系统审查中的数据提取的第二审稿人:两种人工智能工具与人类审稿人的性能比较.
T Helms Andersen1, T M Marcussen1, A D Termannsen1
1Copenhagen University Hospital-Steno Diabetes Center Copenhagen Herlev Denmark.
Cochrane evidence synthesis and methods..
|July 15, 2025
概括
大型语言模型 (LLM) 和人工智能 (AI) 工具在系统审查数据提取方面表现出很高的表现. 人工智能辅助提取可以取代第二个人类审核员,提高效率.
科学领域:
- 医疗信息学 医疗信息学
- 医疗保健中的人工智能
- 证据综合 证据综合
背景情况:
- 系统性审查对于基于证据的医学至关重要,但资源密集.
- 使用人工智能和LLM自动化数据提取提供了一个潜在的解决方案,以减少时间和成本.
- 在各种系统性审查类型中,没有针对人工智能的全面工作流被验证.
研究的目的:
- 评估Elicit和ChatGPT在从期刊文章中提取数据方面的有效性.
- 在系统性审查中,评估人工智能工具作为取代人类数据提取器的替代品.
主要方法:
- 通过人工智能提取的数据,比较了三次系统性审查中的30篇文章中人类提取的数据.
- 引发和ChatGPT提取了人口特征,研究设计和审查特定变量.
- 计算性能指标 (精度,回忆,F1得分) 与人类双重提取作为黄金标准,然后进行错误分析.
主要成果:
- 提取实现了92%的精度,回忆和F1得分;ChatGPT分别实现了91%,89%和90%.
- 人工智能回忆对于研究设计 (Elicit: 100%,ChatGPT: 90%) 和人口特征 (Elicit: 100%,ChatGPT: 97%) 的高.
- 引起和ChatGPT显示了类似的表现,与小 confabulations (4%的数据点).
结论:
- 人工智能工具在数据提取方面表现出与人类审核人员相似的性能,特别是在标准化变量方面.
- 建议采用人工智能辅助的提取工作流程,取代第二个人类提取器.
- 人类审查员可以专注于调和人工智能与人类之间的差异,优化系统审查过程.
更多相关视频
05:47Evidence-based Knowledge Synthesis and Hypothesis Validation: Navigating Biomedical Knowledge Bases via Explainable AI and Agentic Systems
Published on: June 13, 2025
587
04:54Author Spotlight: IntelliSleepScorer — A High-Accuracy, Accessible GUI Software for Automated Sleep Stage Scoring in Mice and its Application in Psychiatric Research
Published on: November 8, 2024
661
相关概念视频
Extraction: Advanced Methods
541
Metal ions can be separated from one another by complexation with organic ligands–the chelating agent– to form uncharged chelates. Here, the chelating agent must contain hydrophobic groups and behave as a weak acid, losing a proton to bind with the metal. Since most organic ligands used in this process are insoluble or undergo oxidation in the aqueous phase, the chelating agent is initially added to the organic phase and extracted into the aqueous phase. The metal-ligand complex is...
541
Non-equilibrium in the Cell
4.9K
An important concept in studying metabolism and energy is that of chemical equilibrium. Most chemical reactions are reversible. They can proceed in both directions, releasing energy into their environment in one direction, and absorbing it from the environment in the other direction. The same is true for the chemical reactions involved in cell metabolism, such as the breaking down and building up of proteins into and from individual amino acids, respectively. Reactants within a closed system...
4.9K
Improving Translational Accuracy
2.7K
2.7K
Multiple Comparison Tests
4.0K
Multiple comparison test, abbreviated as MCT, is a post hoc analysis generally performed after comparing multiple samples with one or more tests. An MCT will help identify a significantly different sample among multiple samples or a factor among multiple factors.
It would be easy to compare two samples using a significance alpha level of 0.05. In other words, there is only one sample pair to be compared. However, it would be difficult to identify a significantly different sample if the number...
It would be easy to compare two samples using a significance alpha level of 0.05. In other words, there is only one sample pair to be compared. However, it would be difficult to identify a significantly different sample if the number...
4.0K
