评估困惑度评分在区分人工智能生成和人类撰写摘要方面的有效性
Alperen Elek1, Hatice Sude Yildiz2, Benan Akca3
1Ege University Faculty of Medicine, Izmir, Turkey (A.E.).
Academic radiology
|February 6, 2025
概括
困惑度得分可以帮助区分人工智能生成的放射学摘要与人类撰写的摘要. 虽然一些人工智能检测工具显示出希望,但需要进一步的研究来验证这些发现的学术写作.
科学领域:
- 放射学 放射学是一门学科.
- 人工智能的人工智能
- 学术出版学术出版公司
背景情况:
- 人工智能 (AI) 在学术写作中的兴起需要方法来检测AI产生的内容.
- 区分人类写的和人工智能生成的放射学摘要对于保持学术完整性至关重要.
研究的目的:
- 评估困惑度评分在区分人类写的与人工智能生成的放射学摘要中的有效性.
- 评估现有的AI检测工具在识别人工智能生成的摘要方面的性能.
主要方法:
- 策划了2021年之前发表的50篇关于神经成像和血管学的PubMed文章.
- 从完整文本中生成AI摘要,并将困惑度得分与原始人类撰写的摘要进行比较.
- 评估了三种人工智能检测工具在分类摘要中的准确性.
主要成果:
- 人类撰写的摘要比人工智能生成的摘要 (中位数21.2) 有更高的困惑度得分 (中位数35.9),AUC为0.7794 (p=0.057).
- 一个人工智能工具在检测中实现了95%的准确性 (AUC=0.8688),而另一个则低于偶然性 (36%的准确性).
结论:
- 困惑度得分显示了检测人工智能生成的摘要的潜力,这表明它可能在识别欺诈性内容方面发挥作用.
- 需要进一步的研究来探索这些发现对学术写作中的AI的影响,并研究替代检测方法.
更多相关视频
07:34Perceptual and Category Processing of the Uncanny Valley Hypothesis' Dimension of Human Likeness: Some Methodological Issues
Published on: June 3, 2013
17.3K
10:26Problem-Solving Before Instruction PS-I: A Protocol for Assessment and Intervention in Students with Different Abilities
Published on: September 11, 2021
3.9K
相关概念视频
Introduction to Cognitive Psychology
409
Cognitive psychology is the field of psychology dedicated to examining how people think. It attempts to explain how and why we think the way we do by studying the interactions among human thinking, emotion, creativity, language, and problem-solving, as well as other cognitive processes. Cognitive psychology studies how information is processed and manipulated in remembering, thinking, and knowing.
This field emerged in the mid-20th century, following a period dominated by behaviorism, which...
This field emerged in the mid-20th century, following a period dominated by behaviorism, which...
409
Reason and Intuition
6.3K
The human brain processes information for decision-making using one of two routes: an intuitive system and a rational system (Epstein, 1994; popularized by Kahneman, 2011 as System 1 and System 2, respectively). The intuitive system is quick, impulsive, and operates with minimal effort, relying on emotions or habits to provide cues for what to do next, while the rational system is logical, analytical, deliberate, and methodical. Research in neuropsychology suggests that the...
6.3K
Stereotype Content Model
13.9K
The Stereotype Content Model (SCM) was first proposed by Susan Fiske and her colleagues (Fiske, Cuddy, Glick & Xu, 2002; see also Fiske, 2012 and Fiske, 2017). The SCM specifies that when someone encounters a new group, they will stereotype them based on two metrics: warmth—or that group’s perceived intent, and how likely they are to provide help or inflict harm—and competence—or their ability to carry out that objective. Depending on the warmth-competence...
13.9K
