人工智能检测工具和人类准确识别不同形式的AI生成的书面内容的能力
Adam Cheng1, Yiqun Lin2, Gabriel Reedy3
1Departments of Pediatrics and Emergency Medicine, Cumming School of Medicine, KidSIM Simulation Program, Alberta Children's Hospital, University of Calgary, 28 Oki Drive NW, Calgary, AB, T3B 6A8, Canada. chenger@me.com.
Advances in simulation (London, England)
|November 22, 2025
概括
人工智能检测工具可以在学术文章中识别人工智能 (AI) 的使用,但显示可变可靠性. 人类对人工智能内容的评估是不可靠的,这强调了在学术出版中需要谨慎使用人工智能检测工具的必要性.
科学领域:
- 学术出版学术出版活动
- 人工智能 (AI) 在研究中的作用
- 学术诚信 学术诚信
背景情况:
- 研究人员越来越多地使用人工智能 (AI),这给医疗出版业带来了挑战.
- 不道德的AI使用可能导致不准确和偏见的学术内容.
- 人工智能检测工具提供了一种方法来评估学术写作中的AI内容.
研究的目的:
- 为了比较三个AI检测工具 (ZeroGPT,PhraslyAI,文法AI检测器) 的性能.
- 评估人工智能检测工具的分数与人工智能在五个使用条件中的人类评估.
- 评估AI检测工具在学术内容中的可靠性和准确性.
主要方法:
- 从"模拟的进步"和"医疗保健模拟"杂志中选择了30篇开放访问文章介绍.
- 研究人员检查了人工智能使用的五个条件:100%的人类,轻量化人工智能编辑,重量化人工智能编辑,人工智能从人类内容中生成,以及100%人工智能生成.
- 人工智能检测工具和人类评估人员评估了材料,并使用ANOVA和ICC分析了结果.
主要成果:
- 人工智能检测工具区分了人工智能使用条件 (p < 0.001),但显示了显著的得分变化 (ICC 0.57-0.95).
- 人类得分准确度低于19%,表现不好于机会.
- 人工智能检测工具的可靠性是一个问题,尽管它们能够区分人工智能使用.
结论:
- 现有的AI检测工具可以区分合理的AI使用,但表现出可变的可靠性.
- 人类对人工智能产生的内容的评估总是不准确的.
- 人工智能检测工具可能有助于识别不道德的人工智能使用,但不应仅仅依赖于它们.
相关概念视频
01:16Non-equilibrium in the Cell
5.3K
An important concept in studying metabolism and energy is that of chemical equilibrium. Most chemical reactions are reversible. They can proceed in both directions, releasing energy into their environment in one direction, and absorbing it from the environment in the other direction. The same is true for the chemical reactions involved in cell metabolism, such as the breaking down and building up of proteins into and from individual amino acids, respectively. Reactants within a closed system...
5.3K
01:14Understanding Deception
146
Deception is a pervasive aspect of human communication. Empirical studies have shown that most individuals engage in some form of deceit on a daily basis, with approximately 20% of social exchanges involving deceptive elements. Lying follows a developmental trajectory, peaking during adolescence and declining with age, possibly due to the maturation of cognitive control and social accountability.Cognitive and Social Factors in Deception DetectionDespite its prevalence, accurately detecting...
146
01:20Introduction to Cognitive Psychology
2.2K
Cognitive psychology is the field of psychology dedicated to examining how people think. It attempts to explain how and why we think the way we do by studying the interactions among human thinking, emotion, creativity, language, and problem-solving, as well as other cognitive processes. Cognitive psychology studies how information is processed and manipulated in remembering, thinking, and knowing.
This field emerged in the mid-20th century, following a period dominated by behaviorism, which...
This field emerged in the mid-20th century, following a period dominated by behaviorism, which...
2.2K