使用xFakeSci学习算法检测ChatGPT假科学
Ahmed Abdeen Hamed1, Xindong Wu2,3
1Complex Adaptive Systems and Computational Intelligence Laboratory, State University of New York at Binghamton, Binghamton, NY, 13902, USA. ahamed1@binghamton.edu.
Scientific reports
|July 14, 2024
概括
这项研究介绍了xFakeSci,这是一种新的算法,可以有效地区分人工智能生成的科学文章和人类撰写的文章. xFakeSci实现了高准确度,为打击假科学的兴起提供了一个至关重要的工具.
科学领域:
- 人工智能的人工智能
- 医疗信息学 医疗信息学
- 科学出版科学出版
背景情况:
- 像ChatGPT这样的生成性AI工具越来越普遍.
- 人工智能产生的内容可能具有与真正的科学文章分开的独特特征.
- 快速工程能够创建AI生成的关于各种疾病的文章.
研究的目的:
- 与科学文章相比,研究人工智能生成的内容的独特行为.
- 开发和验证一个新的算法,xFakeSci,用于区分人工智能产生的科学家撰写的出版物.
- 评估xFakeSci与传统数据挖掘算法的有效性.
主要方法:
- 文章是使用用于癌症,抑郁症和阿尔茨海默氏症的快速工程生成的.
- 该xFakeSci算法被训练在来自AI生成和真实文章的网络模型上.
- 使用数据驱动的启发式 (近距离,比率) 的校准步骤被纳入,以减轻过度拟合.
- 分类涉及每种条件的300篇文章,与真实的PubMed摘要进行平衡.
- 性能与支持向量机,回归和天真贝叶斯算法进行了评估.
主要成果:
- xFakeSci算法实现了高F1分数,从80%到94%不等.
- 经典的数据挖掘算法得分明显较低的F1值,在38%和52%之间.
- xFakeSci的增强性能归因于其校准步骤和近距离启发式.
结论:
- 人工智能产生的内容与科学出版物有明显的差异.
- xFakeSci算法在识别和打击人工智能生成的假科学方面取得了重大进展.
- 开发的算法在区分合成科学文献方面表现出卓越的性能.
相关概念视频
02:55Synthetic Biology
4.4K
Synthetic biology is an interdisciplinary science that involves using principles from disciplines such as engineering, molecular biology, cell biology, and systems biology. It involves remodeling existing organisms from nature or constructing completely new synthetic organisms for applications such as protein or enzyme production, bioremediation, value-added macromolecule production, and the addition of desirable traits to crops, to name a few.
Golden rice
Golden rice is a genetically modified...
Golden rice
Golden rice is a genetically modified...
4.4K
01:10Detection of Black Holes
1.7K
Although black holes were theoretically postulated in the 1920s, they remained outside the domain of observational astronomy until the 1970s.
Their closest cousins are neutron stars, which are composed almost entirely of neutrons packed against each other, making them extremely dense. A neutron star has the same mass as the Sun but its diameter is only a few kilometers. Therefore, the escape velocity from their surface is close to the speed of light.
Not until the 1960s, when the first neutron...
Their closest cousins are neutron stars, which are composed almost entirely of neutrons packed against each other, making them extremely dense. A neutron star has the same mass as the Sun but its diameter is only a few kilometers. Therefore, the escape velocity from their surface is close to the speed of light.
Not until the 1960s, when the first neutron...
1.7K
01:00Detection of Gross Error: The Q Test
7.1K
When one or more data points appear far from the rest of the data, there is a need to determine whether they are outliers and whether they should be eliminated from the data set to ensure an accurate representation of the measured value. In many cases, outliers arise from gross errors (or human errors) and do not accurately reflect the underlying phenomenon. In some cases, however, these apparent outliers reflect true phenomenological differences. In these cases, we can use statistical methods...
7.1K
01:12Trial and Error and Algorithm
546
A problem-solving strategy is a plan of action used to find a solution. Different strategies have distinct action plans. Trial and error involves trying different solutions until one works. For instance, to fix a broken printer, you might check ink levels, ensure the paper tray isn't jammed, and verify the printer's connection to your laptop. This method can be time-consuming but is commonly used. Thomas Edison, for example, used trial and error to find a suitable filament for the light...
546