在产科系统审查的研究选择中,ChatGPT和Claude的表现如何?
Suppachai Insuk1, Kansak Boonpattharatthiti2,3, Chimbun Booncharoen1
1Faculty of Pharmaceutical Sciences, Naresuan University, Phitsanulok, Thailand.
像ChatGPT和Claude这样的生成人工智能模型在系统审查研究选择中显示出希望. 虽然人类研究人员在初步选中表现稍好,但人工智能显示出强大的潜力,特别是在全文审查中,接近人类水平的表现.
科学领域:
- 医疗信息学
- 医疗保健中的人工智能
- 基于证据的医学
背景情况:
- 对系统审查效率的研究越来越多.
- 目前关于AI查性能的证据是不确定的.
- 不同的人工智能模型之间存在有限的直接比较.
研究的目的:
- 在产科系统性审查研究中评估ChatGPT-4o和克劳德3.5索内特.
- 将人工智能性能与人类研究人员进行比较.
主要方法:
- 系统的文献搜索 (PubMed,EMBASE,Cochrane CENTRAL,EBSCO开放论文) 进行.
- 使用基于提示的方法进行标题/摘要选,将AI和初级研究人员与经验丰富的研究人员进行比较.
- 使用AI的长短提示策略进行全文审查.
- 性能指标:准确度,灵敏度,精度,F1分数,负预测值 (NPV).
主要成果:
- 标题/摘要查:人类研究人员的精度最高 (0.9593) 和F1得分最高 (0.3853),其次是Claude和ChatGPT. 所有这些都显示出高净值.
- 全文选:使用短提示的ChatGPT获得了最高的准确度 (0.904),F1得分 (0.90) 和NPV (1.00),超过了克劳德和人类研究人员.
结论:
- 生成型人工智能模型在系统审查研究选择中显示了接近人类水平的性能.
- 人工智能显示出显著的潜力,特别是在全文审查阶段.
- 建议在各个领域对人工智能整合进行进一步的研究.
更多相关视频
09:03Semiconductor Sequencing for Preimplantation Genetic Testing for Aneuploidy
Published on: August 25, 2019
09:51Model Surgical Training: Skills Acquisition in Fetoscopic Laser Photocoagulation of Monochorionic Diamniotic Twin Placenta Using Realistic Simulators
Published on: March 21, 2018
相关概念视频
Clinical Trials: Overview
Study Design in Statistics
Does aspirin reduce the risk of heart attacks? Is one brand of fertilizer more effective at growing roses than another? Is fatigue as dangerous to a driver as the influence of alcohol? Questions like these are answered using randomized experiments with proper...
Comparing the Survival Analysis of Two or More Groups
Types of Biopharmaceutical Studies: Controlled and Non-Controlled Approaches
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast,...
Clinical Trials
There are four phases in a clinical trial. A phase one...
Gonadal and Placental Hormones
In males, testosterone is the primary gonadal androgen. It plays a central role in the maturation of male reproductive organs — the penis and testes. Additionally, testosterone is instrumental in the development of secondary sexual characteristics — a deep voice as well as facial and pubic hair...
