人工智能驱动的证据合成:用大型语言模型随机对照试验的数据提取
Jiayi Liu1,2, Honghao Lai1,2, Weilong Zhao1,2
1Department of Health Policy and Health Management, School of Public Health, Lanzhou University, Lanzhou, China.
International journal of surgery (London, England)
|February 4, 2025
概括
大型语言模型 (LLM) 的结构化提示显著改善了用于证据合成的数据提取. 这种方法在随机对照试验中实现了94.77%的准确性,增强了系统审查.
科学领域:
- 医疗信息学 医疗信息学
- 医疗保健中的人工智能
- 证据综合方法论的方法论
背景情况:
- 大型语言模型 (LLM) 具有提高证据合成效率的潜力,特别是在数据提取方面.
- 目前的LLM数据提取提示是有限的,通常专注于标准项目,不适应多样化的研究需求.
- 系统性审查和证据综合需要准确有效地从随机对照试验 (RCT) 等研究中提取数据.
研究的目的:
- 开发和评估LLM从随机对照试验 (RCT) 提取数据的结构化提示.
- 评估使用LLM与结构化提示在证据合成中的可行性和准确性.
- 为了确定LLM驱动的数据提取系统审查方法的效率.
主要方法:
- 开发了用于LLM数据提取的6个Cochrane手册领域的58个项目,全面的结构化提示.
- 使用Claude (Claude-2) 作为快速测试的LLM平台.
- 测试了从已发表的Cochrane评论中随机选择的10个RCT的结构化提示.
主要成果:
- 实现了94.77%的高整体正确数据提取率 (95% CI: 93.66%至95.73%).
- 证明了强大的域特定精度,从77.97%到100%不等.
- 展示了显著的效率,每次RCT的平均提取时间仅为88秒.
结论:
- 当由结构化提示指导时,LLM是可行的和有价值的工具,用于增强证据合成中的数据提取.
- 这种方法代表了系统审查方法的重大进步,提高了准确性和效率.
- 这些发现支持将LLM驱动工具集成到证据综合工作流中,以实现更强大的研究.
相关概念视频
Data Collection by Experiments
23.8K
Data collection is a systematic method of obtaining, observing, measuring, and analyzing accurate information. An experimental study is a standard method of data collection that involves the manipulation of the samples by applying some form of treatment prior to data collection. It refers to manipulating one variable to determine its changes on another variable. The sample subjected to treatment is known as “experimental units.”
An example of the experimental method is a public...
An example of the experimental method is a public...
23.8K
Randomized Experiments
6.7K
The randomization process involves assigning study participants randomly to experimental or control groups based on their probability of being equally assigned. Randomization is meant to eliminate selection bias and balance known and unknown confounding factors so that the control group is similar to the treatment group as much as possible. A computer program and a random number generator can be used to assign participants to groups in a way that minimizes bias.
Simple randomization
Simple...
Simple randomization
Simple...
6.7K
Hazard Ratio
78
The hazard ratio (HR) is a widely used measure in clinical trials to compare the risk of events, such as death or disease recurrence, between two groups over time. It reflects the ratio of hazard rates—the instantaneous risk of the event occurring—between a treatment group and a control group. This measure provides valuable insights into the relative effectiveness of a treatment by assessing how the risk of an event differs between the two groups.
For example, in a clinical trial...
For example, in a clinical trial...
78
What is an Experiment?
10.3K
An experiment is a planned activity carried out under controlled conditions. The purpose of an experiment is to investigate the relationship between two variables. When one variable causes change in another, we call the first variable the explanatory or independent variable. The affected variable is called the response or dependent variable. In a randomized experiment, the researcher manipulates values of the explanatory variable and measures the resulting changes in the response variable. The...
10.3K
Types of Biopharmaceutical Studies: Controlled and Non-Controlled Approaches
114
Biopharmaceutical studies constitute a vital field aiming to enhance drug delivery methods and refine therapeutic approaches, drawing upon diverse interdisciplinary knowledge. In research methodologies, the choice between controlled and non-controlled studies significantly influences the study's reliability and accuracy.
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast,...
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast,...
114
Factors Affecting Drug Response: Overview
1.8K
When it comes to infants and young children, they are typically administered smaller doses of medication in comparison to adults. This is primarily because their organ functions still need to fully develop, meaning their bodies are not as efficient at metabolizing or eliminating drugs. Additionally, their blood-brain barrier is more permeable than in adults. As a result, high concentrations of drugs can easily penetrate the central nervous system (CNS), potentially leading to neurological...
1.8K


