由ChatGPT生成的文献引用中的伪造和错误
William H Walters1, Esther Isabelle Wilder2,3
1Mary Alice & Tom O'Malley Library, Manhattan College, Riverdale, NY, USA. william.walters@manhattan.edu.
Scientific reports
|September 7, 2023
概括
与ChatGPT-3.5相比,ChatGPT-4显著减少了伪造的引用,但这两种AI语言模型在学术文献评论中仍然会产生错误.
科学领域:
- 人工智能的人工智能
- 图书统计学 图书统计学
- 学术传播学术交流
背景情况:
- 像ChatGPT这样的大型语言模型 (LLM) 提供了高效的文本生成,但容易出现事实上的不准确性,称为幻觉.
- 一种关键的幻觉类型是伪造不存在的学术引用,破坏研究完整性.
研究的目的:
- 评估通过ChatGPT-3.5和ChatGPT-4.5生成的编造文献引用的流行情况.
- 在人工智能生成的文献评论中评估非编造引用的准确性和遵守引用标准.
主要方法:
- 聊天GPT-3.5和聊天GPT-4被要求生成42个不同主题的文献评论.
- 在多个数据库中,从84篇生成的论文中总共引用了636个引文,系统地对其伪造和准确性进行了验证.
- 美国心理学会 (APA) 引用格式的遵守也被评估.
主要成果:
- 在55%的实例中,ChatGPT-3.5产生了伪造的引用,而43%的真实引用包含错误.
- 聊天GPT-4显示了显著的改进,只有18%的引用是伪造的,24%的真实引用包含错误.
- 尽管有进步,但这两种模型都表现出大量的引用不准确性.
结论:
- 在生成准确的文献引用方面,ChatGPT-4比ChatGPT-3.5有了显著的改进.
- 人工智能引文的持续挑战需要在学术和研究背景下进行仔细的验证.
- 需要进一步开发,以提高学术写作LLM的可靠性.
相关概念视频
Random and Systematic Errors
11.0K
Scientists always try their best to record measurements with the utmost accuracy and precision. However, sometimes errors do occur. These errors can be random or systematic. Random errors are observed due to the inconsistency or fluctuation in the measurement process, or variations in the quantity itself that is being measured. Such errors fluctuate from being greater than or less than the true value in repeated measurements. Consider a scientist measuring the length of an earthworm using a...
11.0K
Systematic Error: Methodological and Sampling Errors
1.5K
In the case of systematic errors, the sources can be identified, and the errors can be subsequently minimized by addressing these sources. According to the source, systematic errors can be divided into sampling, instrumental, methodological, and personal errors.
Sampling errors originate from improper sampling methods or the wrong sample population. These errors can be minimized by refining the sampling strategy. Defective instruments or faulty calibrations are the sources of instrumental...
Sampling errors originate from improper sampling methods or the wrong sample population. These errors can be minimized by refining the sampling strategy. Defective instruments or faulty calibrations are the sources of instrumental...
1.5K
Accuracy and Errors in Hypothesis Testing
220
Hypothesis testing is a fundamental statistical tool that begins with the assumption that the null hypothesis H0 is true. During this process, two types of errors can occur: Type I and Type II. A Type I error refers to the incorrect rejection of a true null hypothesis, while a Type II error involves the failure to reject a false null hypothesis.
In hypothesis testing, the probability of making a Type I error, denoted as α, is commonly set at 0.05. This significance level indicates a 5%...
In hypothesis testing, the probability of making a Type I error, denoted as α, is commonly set at 0.05. This significance level indicates a 5%...
220
Detection of Gross Error: The Q Test
6.2K
When one or more data points appear far from the rest of the data, there is a need to determine whether they are outliers and whether they should be eliminated from the data set to ensure an accurate representation of the measured value. In many cases, outliers arise from gross errors (or human errors) and do not accurately reflect the underlying phenomenon. In some cases, however, these apparent outliers reflect true phenomenological differences. In these cases, we can use statistical methods...
6.2K
Types of Errors: Detection and Minimization
1.7K
Error is the deviation of the obtained result from the true, expected value or the estimated central value. Errors are expressed in absolute or relative terms.
Absolute error in a measurement is the numerical difference from the true or central value. Relative error is the ratio between absolute error and the true or central value, expressed as a percentage.
Errors can be classified by source, magnitude, and sign. There are three types of errors: systematic, random, and gross.
Systematic or...
Absolute error in a measurement is the numerical difference from the true or central value. Relative error is the ratio between absolute error and the true or central value, expressed as a percentage.
Errors can be classified by source, magnitude, and sign. There are three types of errors: systematic, random, and gross.
Systematic or...
1.7K
Random Error
923
Random or indeterminate errors originate from various uncontrollable variables, such as variations in environmental conditions, instrument imperfections, or the inherent variability of the phenomena being measured. Usually, these errors cannot be predicted, estimated, or characterized because their direction and magnitude often vary in magnitude and direction even during consecutive measurements. As a result, they are difficult to eliminate. However, the aggregate effect of these errors can be...
923


