使用GPT-4o和o3进行系统审查的数据提取过程的自动化
Yuki Kataoka1,2,3,4, Tomohiro Takayama5,6, Keisuke Yoshimura5
1Department of Internal Medicine, Kyoto Min-iren Asukai Hospital, Kyoto, Japan.
Research synthesis methods
|February 2, 2026
概括
我们开发了一个自动数据提取系统,使用GPT-4o进行系统性审查. 虽然它对字符串数据有效,但对于数字数据提取,它需要人类监督.
科学领域:
- 人工智能的人工智能
- 生物医学信息学 生物医学信息学
- 卫生科学 卫生科学 卫生科学
背景情况:
- 在系统性审查 (SRs) 中自动化数据提取 (DE) 对效率至关重要.
- 现有的自动化DE方法通常需要大量的手动输入.
- 大型语言模型 (LLM) 显示了改善DE自动化的潜力.
研究的目的:
- 开发和评估一个开源的,完全自动化的DE系统,使用GPT-4o用于SRs.
- 评估系统在各种数据集上的性能,包括字符串和数值变量.
- 为了比较自动化系统的精度与手动提取.
主要方法:
- 开发了一个开源的DE系统,利用GPT-4o在提取过程中没有人类干预.
- 在290个随机对照试验 (RCT) 的数据集上训练和优化了系统.
- 在两个独立的数据集上验证了系统,包括更新的搜索和单独发表的研究.
主要成果:
- 该系统实现了高变量检测全面性 (93.5%),但变量准确性在字符串 (较高) 和数字 (较低) 数据之间有所不同.
- 最初的外部验证显示,GPT-4o的平均精度为84.4%.
- 调整后的DE方法在外部数据集上提高了准确度至96.3%,与手动提取相比.
结论:
- 基于GPT-4o的系统显示出在SR中协助字符串变量DE的前景.
- 该系统目前无法完全取代数字数据提取的人类审查员.
- 需要进一步的研究,以确定这个自动化DE系统在各种审查环境中的更广泛适用性.
相关概念视频
Review and Preview
8.4K
In statistics, several tools are used to interpret the data. Measures of central tendency represent the characteristics of the data, such as mean, median, and mode. Additionally, measures of variance like standard deviation and range are used to find the spread of data from the mean. Relative standing measures the distance between data locations. Commonly used measures of relative standings are percentile, z score, and quartiles.
Percentiles are a type of fractile that partition data into...
Percentiles are a type of fractile that partition data into...
8.4K
Review and Preview
11.3K
Data are individual items of information obtained from a population or sample. Data may be classified as qualitative (categorical), quantitative continuous, or quantitative discrete. Because it is not practical to measure the entire population in a study, researchers use samples to represent the population. A random sample is a representative group from the population chosen by using a method that gives each individual in the population an equal chance of being included in the sample. Random...
11.3K
Random and Systematic Errors
14.9K
Scientists always try their best to record measurements with the utmost accuracy and precision. However, sometimes errors do occur. These errors can be random or systematic. Random errors are observed due to the inconsistency or fluctuation in the measurement process, or variations in the quantity itself that is being measured. Such errors fluctuate from being greater than or less than the true value in repeated measurements. Consider a scientist measuring the length of an earthworm using a...
14.9K
Systematic Sampling Method
13.3K
Sampling is a technique to select a portion (or subset) of the larger population and study that portion (the sample) to gain information about the population. Data are the result of sampling from a population. The sampling method ensures that samples are drawn without bias and accurately represent the population. Because measuring the entire population in a study is not practical, researchers use samples to represent the population of interest.
Systematic sampling is one of the simplest methods...
Systematic sampling is one of the simplest methods...
13.3K
Propagation of Uncertainty from Systematic Error
1.5K
The atomic mass of an element varies due to the relative ratio of its isotopes. A sample's relative proportion of oxygen isotopes influences its average atomic mass. For instance, if we were to measure the atomic mass of oxygen from a sample, the mass would be a weighted average of the isotopic masses of oxygen in that sample. Since a single sample is not likely to perfectly reflect the true atomic mass of oxygen for all the molecules of oxygen on Earth, the mass we obtain from this...
1.5K
Systematic Error: Methodological and Sampling Errors
11.0K
In the case of systematic errors, the sources can be identified, and the errors can be subsequently minimized by addressing these sources. According to the source, systematic errors can be divided into sampling, instrumental, methodological, and personal errors.
Sampling errors originate from improper sampling methods or the wrong sample population. These errors can be minimized by refining the sampling strategy. Defective instruments or faulty calibrations are the sources of instrumental...
Sampling errors originate from improper sampling methods or the wrong sample population. These errors can be minimized by refining the sampling strategy. Defective instruments or faulty calibrations are the sources of instrumental...
11.0K


