大型语言模型在选引文中的性能
Takehiko Oami1, Yohei Okada2,3, Taka-Aki Nakada1
1Department of Emergency and Critical Care Medicine, Chiba University Graduate School of Medicine, Chiba, Japan.
JAMA network open
|July 8, 2024
概括
大型语言模型 (LLM) 在系统性审查中显示出引用选的前景,显示出可接受的灵敏度和高特异性,同时与传统方法相比显著减少选时间.
科学领域:
- 医疗信息学 医疗信息学
- 系统审查方法论 系统审查方法论
- 医疗保健中的人工智能
背景情况:
- 系统性审查对于以证据为基础的医学至关重要,但耗时.
- 引用选是系统审查过程中的一个主要瓶.
- 大型语言模型 (LLM) 提供了自动化和提高文献评论效率的潜力.
研究的目的:
- 评估大型语言模型 (LLM) 的准确性和效率,用于标题和摘要引用选.
- 将LLM辅助选与传统的手动选方法进行比较.
主要方法:
- 一项前性诊断研究评估了GPT-4 Turbo在5个临床问题的毒指南中进行引用查.
- 根据预先定义的纳入/排除标准,LLM选决策是基于.
- 性能指标 (灵敏度,特异性) 和时间效率与手动查进行了比较.
主要成果:
- 在初级分析中,LLM辅助查获得了0.75的灵敏度和0.99的特异性.
- 随时后的提示性修改使灵敏度提高到0.91,同时保持高特异性 (0.98).
- 通过LLM选,100项研究的处理时间从17.2分钟减少到1.3分钟.
结论:
- 通过LLM辅助的引用选显示了可接受的灵敏度和高特异性.
- 这种方法大大减少了在系统性审查中对文献选所需的时间.
- 在系统性审查中,LLM有潜力提高效率并减少工作量.
相关概念视频
Sample Size Calculation
3.3K
Knowledge of the sample size is the first requirement to conduct random sampling or an experiment. The sample size is the total number of units, observations, or groups (in some cases) used to get the data to estimate a population parameter. As the name suggests, the sample size is that of the sample drawn from the population and differs from the population size.
The sample size for the given experiment or sampling effort is fundamental to any study design. Sample size decides the number of...
The sample size for the given experiment or sampling effort is fundamental to any study design. Sample size decides the number of...
3.3K
Improving Translational Accuracy
2.6K
2.6K
Goodness-of-Fit Test
3.3K
The goodness-of-fit test is a type of hypothesis test which determines whether the data "fits" a particular distribution. For example, one may suspect that some anonymous data may fit a binomial distribution. A chi-square test (meaning the distribution for the hypothesis test is chi-square) can be used to determine if there is a fit. The null and alternative hypotheses may be written in sentences or stated as equations or inequalities. The test statistic for a goodness-of-fit test is given as...
3.3K
Higher Mental Functions of the Brain: Language
782
Language is a system of communication that allows the expression of thoughts, ideas, and feelings. The brain processes language in both hemispheres.
Language formation and comprehension take place in the dominant hemisphere. The dominant hemisphere is responsible for understanding the meaning of spoken, written, or sign language, as well as the ability to communicate. For most people, the left hemisphere is the dominant one. The right hemisphere, then, gives tone and emotional context to the...
Language formation and comprehension take place in the dominant hemisphere. The dominant hemisphere is responsible for understanding the meaning of spoken, written, or sign language, as well as the ability to communicate. For most people, the left hemisphere is the dominant one. The right hemisphere, then, gives tone and emotional context to the...
782
Detection of Gross Error: The Q Test
6.0K
When one or more data points appear far from the rest of the data, there is a need to determine whether they are outliers and whether they should be eliminated from the data set to ensure an accurate representation of the measured value. In many cases, outliers arise from gross errors (or human errors) and do not accurately reflect the underlying phenomenon. In some cases, however, these apparent outliers reflect true phenomenological differences. In these cases, we can use statistical methods...
6.0K
Quantifying and Rejecting Outliers: The Grubbs Test
1.5K
Sometimes, a data set can have a recorded numerical observation that greatly deviates from the rest of the data. Assuming that the data is normally distributed, a statistical method called the Grubbs test can be used to determine whether the observation is truly an outlier. To perform a two-tailed Grubbs test, first, calculate the absolute difference between the outlier and the mean. Then, calculate the ratio between this difference and the standard deviation of the sample. This...
1.5K


