在公共政策文献综述中使用GPT-4进行标题和摘要选:可行性研究
Max Rubinstein1, Sean Grant1,2, Beth Ann Griffin1
1RAND Pittsburgh Pennsylvania USA.
Cochrane evidence synthesis and methods..
|July 14, 2025
概括
大型语言模型 (LLM) 有效地选了公共政策文献进行审查,大大减少了需要手动评估的文章数量. 在识别需要移除的物品时,GPT-4的虚假排除率非常低.
科学领域:
- 图书统计学 图书统计学
- 研究中的人工智能.
- 公共政策分析 公共政策分析
背景情况:
- 系统的文献评论需要对标题和摘要进行广泛的选.
- 大型语言模型 (LLM) 提供了自动化部分文献审查过程的潜力.
- 评估LLM驱动选的准确性对于可靠的系统审查至关重要.
研究的目的:
- 评估GPT-4在选标题和摘要中的表现,以进行公共政策文献综述.
- 为了确定GPT-4的错误排除率,在推排除的文章时.
- 评估LLM辅助文学选所提供的潜在时间和成本节约.
主要方法:
- GPT-4被用来选43,480篇文章,用于对阿片类药物危机政策的文献综述.
- 文章被导出到一个CSV文件,GPT-4建议基于标题和摘要的排除.
- 进行了初步和最终测试,以确定错误排除率,并设置了足够性能的10%门.
主要成果:
- GPT-4建议不包括96% (41,742) 的物品.
- 该研究发现,错误排除率为0.00 [0.00,0.05],其中不到1%的被排除的文章被错误删除.
- 在识别包含的文章方面,GPT-4的表现很差,其中65%没有排除的实际上是无关紧要的.
结论:
- GPT-4在推被排除的文章方面表现出高度准确性,从而节省了大量的时间和成本.
- 通过过不相关的文章,LLM可以大幅减少系统审查中的工作量.
- 系统审查者必须在将LLM整合到他们的工作流程之前进行严格的绩效评估,以确保质量标准得到满足.
相关概念视频
01:10Pre-Procedural Guidelines for Assessing Blood Pressure
631
Accurate blood pressure assessment is crucial for diagnosing and managing various health conditions. To ensure the reliability of these measurements, healthcare professionals must adhere to standardized pre-procedural guidelines. These guidelines enhance patient safety and improve the overall quality of healthcare. The following steps are essential for obtaining accurate and consistent blood pressure readings, from using the appropriate tools to ensuring effective communication with the...
631
01:23Types of Biopharmaceutical Studies: Controlled and Non-Controlled Approaches
178
Biopharmaceutical studies constitute a vital field aiming to enhance drug delivery methods and refine therapeutic approaches, drawing upon diverse interdisciplinary knowledge. In research methodologies, the choice between controlled and non-controlled studies significantly influences the study's reliability and accuracy.
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast,...
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast,...
178
01:11Guidelines for Writing Outcome
3.0K
When developing expected outcomes for a patient care plan, the nurse should adhere to the following recommendations:
Patient outcomes reflect the patient's response to the goal rather than what the nurse aims to achieve. Terminology should be observable and measurable to avoid the reader's interpretation. The desired outcome should be realistic and achievable in the designated care timeframe. Expected outcomes should align with adjunctive therapies. The outcome should enhance care...
Patient outcomes reflect the patient's response to the goal rather than what the nurse aims to achieve. Terminology should be observable and measurable to avoid the reader's interpretation. The desired outcome should be realistic and achievable in the designated care timeframe. Expected outcomes should align with adjunctive therapies. The outcome should enhance care...
3.0K
01:11Clinical Trials: Overview
3.4K
Clinical development focuses on how the drug will interact with the human body and encompasses four key phases of clinical trials, each serving a specific purpose in assessing the safety and effectiveness of new drugs. These phases overlap and build upon one another. Phase I involves a small group of healthy volunteers (typically 20-80 individuals) or, in cases where significant toxicity is expected, patients with the targeted disease, such as cancer or AIDS. The volunteers are tested for...
3.4K
01:20Study Designs in Epidemiology
422
Epidemiological study designs are fundamental tools for investigating the distribution, determinants, and control of health conditions in populations. They help researchers understand the relationships between exposures and outcomes, and they broadly fall into two categories: "observational" and "experimental" studies.
Observational studies are those where the researcher does not intervene but rather observes natural variations. They include cross-sectional, cohort, and...
Observational studies are those where the researcher does not intervene but rather observes natural variations. They include cross-sectional, cohort, and...
422
01:03Drug Control Governance: Regulatory Bodies and Their Impact
254
Drug control governance involves the oversight and regulation of pharmaceuticals to ensure their safety and efficacy while preventing illegal drug use and trafficking. Regulatory bodies, including the US Food and Drug Administration (FDA) and the European Union's European Medicines Agency (EMA), play a central role in this process. These agencies evaluate the safety and efficacy of drugs before they can be marketed. They fund clinical trials and assess the benefits and risks associated with...
254