通过生成人工智能增强系统文献审查:开发,应用和绩效评估.
Ying Li1, Surabhi Datta2, Majid Rastegar-Mojarad2
1Regeneron Pharmaceuticals, Inc., Tarrytown, NY 10591, United States.
概括
本研究介绍了一种人工智能辅助的系统,用于医疗技术评估 (HTA) 中的系统文献审查 (SLR). 大型语言模型 (LLM) 系统简化了审查,减少了时间和成本,同时改善了证据生成.
科学领域:
- 人工智能在卫生技术评估中的应用
- 系统性文献审查自动化 自动化
- 大型语言模型应用程序
背景情况:
- 系统性文献审查 (SLR) 对于医疗技术评估 (HTA) 至关重要,但耗时且资源密集.
- 目前的单反相机方法面临效率和人为错误的可能性方面的挑战.
- 集成先进的人工智能,特别是大型语言模型 (LLM),为这些挑战提供了潜在的解决方案.
研究的目的:
- 开发和验证LLM辅助系统,用于在HTA提交中进行SLR.
- 评估LLM系统在抽象选,推理生成和数据提取方面的表现.
- 评估系统能够简化SLR流程并增强证据生成的能力.
主要方法:
- 开发了一个由五个模块组成的系统,包括文献搜索设置,协议设置 (PICOS标准),LLM辅助抽象选,LLM辅助数据提取和数据总结.
- 实现了人为循环设计,以根据审查员和LLM的分歧实时调整PICOS标准.
- 该系统使用四个数据集进行了评估,包括复发性和耐火性多发性骨髓瘤 (RRMM) 和晚期黑色素瘤.
主要成果:
- 在抽象选中,LLM系统表现出高性能,平均灵敏度为90%,F1得分为82.
- 该系统在提供排除理由方面取得了高准确性 (高达97%) 和数据提取方面获得了93的F1得分.
- 人类审查者和基于LLM的结果之间观察到实质性的一致性 (科恩的 κ = 0.71).
结论:
- 开发的LLM辅助SLR系统显示了简化HTA提交的巨大潜力.
- 该系统的人在循环设计和依赖GPT-4的上下文学习消除了手动注释训练数据的需要.
- 这种人工智能驱动的方法可以减少单反相机的时间,成本和人为错误,从而改善HTA的证据生成.
相关概念视频
Inductive Reasoning
59.8K
Inductive reasoning is a form of logical thinking that uses related observations to arrive at a general conclusion. It is uncertain and operates in degrees to which the conclusions are credible. As such, inductive arguments can be weak or strong, rather than valid or invalid, and conclusions can be used to formulate testable, falsifiable hypotheses.
Inductive reasoning is common in descriptive science. A life scientist makes observations and records them. This data can be qualitative or...
Inductive reasoning is common in descriptive science. A life scientist makes observations and records them. This data can be qualitative or...
59.8K
Genetic Lingo
99.2K
Overview
99.2K
What is Genetic Engineering?
73.1K
Overview
73.1K
Gene Therapy
25.1K
Gene therapy is a technique where a gene is inserted into a person’s cells to prevent or treat a serious disease. The added gene may be a healthy version of the gene that is mutated in the patient, or it could be a different gene that inactivates or compensates for the patient’s disease-causing gene. For example, in patients with severe combined immunodeficiency (SCID) due to a mutation in the gene for the enzyme adenosine deaminase, a functioning version of the gene can be...
25.1K
Neural Regulation
39.1K
Digestion begins with a cephalic phase that prepares the digestive system to receive food. When our brain processes visual or olfactory information about food, it triggers impulses in the cranial nerves innervating the salivary glands and stomach to prepare for food.
39.1K
Non-equilibrium in the Cell
4.1K
An important concept in studying metabolism and energy is that of chemical equilibrium. Most chemical reactions are reversible. They can proceed in both directions, releasing energy into their environment in one direction, and absorbing it from the environment in the other direction. The same is true for the chemical reactions involved in cell metabolism, such as the breaking down and building up of proteins into and from individual amino acids, respectively. Reactants within a closed system...
4.1K
