评估ChatGPT的定制版本,用于系统审查健康研究中的数据提取:开发和可用性研究
Jayden Sercombe1, Zachary Bryant1, Jack Wilson1
1The Matilda Centre for Research in Mental Health and Substance Use, University of Sydney, Jane Foss Russell Building (G02), Level 6, Sydney, 2006, Australia, 612 8627 9380.
JMIR formative research
|August 11, 2025
概括
使用定制的生成预训练变压器 (GPT) 自动化系统审查数据提取大大减少了时间,并保持了研究特征的高精度. 这种人工智能工具为卫生研究人员提供了一个有前途的解决方案,以简化数据提取过程.
科学领域:
- 健康科学 卫生科学 卫生科学
- 人工智能的人工智能
- 生物医学研究生物医学研究
背景情况:
- 系统性审查对健康科学至关重要,但耗时且容易出现错误.
- 数据提取阶段是系统审查中的一个关键瓶.
- 人工智能的进步,特别是大型语言模型 (LLM),为自动化提供了潜力.
研究的目的:
- 开发和评估一个定制的生成预训练变压器 (GPT),用于在健康研究系统性审查中自动提取数据.
- 与人类审查员相比,评估定制GPT的效率和准确性.
主要方法:
- 使用 OpenAI 的 GPT Builder 和 RISEN 框架进行快速工程,开发了一个定制的 GPT,系统审查提取器 Pro.
- 通过从两个系统性审查中抽取的20项研究样本来评估性能.
- 对于各种数据子部分,计算了GPT输出和人类审核者之间的一致率.
主要成果:
- 定制GPT将提取时间从每项研究 (人类) 的36分钟缩短到26.6秒 (GPT生成) 加13分钟 (人类审查).
- 总体而言,达成协议率很高:审查1的91.45%和审查2的89.31%.
- 在提取研究和参与者特征方面,GPT表现出色,但对方法和统计数据的准确性较低.
结论:
- 定制的GPT显著减少了数据提取时间,并显示出高精度,特别是在研究和参与者特征方面.
- 这种人工智能工具为研究人员提供了一个可行的选择,以减少系统审查中的资源需求.
- 建议进行进一步的研究,以评估可靠性,不同类型审查的性能和提取统计数据的准确性. 该工具是开放访问的.
相关概念视频
Analysis of Population Pharmacokinetic Data
383
Analysis of population pharmacokinetic data involves studying the behavior of drugs within diverse populations to understand their pharmacokinetic parameters. Traditional pharmacokinetic methods typically involve collecting samples from a few individuals and estimating these parameters. While these methods are commonly used, they have limitations in capturing the variability in drug response among individuals or heterogeneous populations. Population pharmacokinetics is employed to address these...
383
Types of Biopharmaceutical Studies: Controlled and Non-Controlled Approaches
174
Biopharmaceutical studies constitute a vital field aiming to enhance drug delivery methods and refine therapeutic approaches, drawing upon diverse interdisciplinary knowledge. In research methodologies, the choice between controlled and non-controlled studies significantly influences the study's reliability and accuracy.
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast,...
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast,...
174
Study Designs in Epidemiology
418
Epidemiological study designs are fundamental tools for investigating the distribution, determinants, and control of health conditions in populations. They help researchers understand the relationships between exposures and outcomes, and they broadly fall into two categories: "observational" and "experimental" studies.
Observational studies are those where the researcher does not intervene but rather observes natural variations. They include cross-sectional, cohort, and...
Observational studies are those where the researcher does not intervene but rather observes natural variations. They include cross-sectional, cohort, and...
418
Comparing the Survival Analysis of Two or More Groups
286
Survival analysis is a cornerstone of medical research, used to evaluate the time until an event of interest occurs, such as death, disease recurrence, or recovery. Unlike standard statistical methods, survival analysis is particularly adept at handling censored data—instances where the event has not occurred for some participants by the end of the study or remains unobserved. To address these unique challenges, specialized techniques like the Kaplan-Meier estimator, log-rank test, and...
286
Pre-Procedural Guidelines for Assessing Blood Pressure
627
Accurate blood pressure assessment is crucial for diagnosing and managing various health conditions. To ensure the reliability of these measurements, healthcare professionals must adhere to standardized pre-procedural guidelines. These guidelines enhance patient safety and improve the overall quality of healthcare. The following steps are essential for obtaining accurate and consistent blood pressure readings, from using the appropriate tools to ensuring effective communication with the...
627
Study Design in Statistics
8.5K
A study design is a set of techniques that allow a researcher to collect and analyze data from different variables defined for a specific research problem. Statistics is commonly for effective study design and more robust experiments,
Does aspirin reduce the risk of heart attacks? Is one brand of fertilizer more effective at growing roses than another? Is fatigue as dangerous to a driver as the influence of alcohol? Questions like these are answered using randomized experiments with proper...
Does aspirin reduce the risk of heart attacks? Is one brand of fertilizer more effective at growing roses than another? Is fatigue as dangerous to a driver as the influence of alcohol? Questions like these are answered using randomized experiments with proper...
8.5K


