人工智能检测软件的准确性 居民个人陈述 个人陈述
Thomas Pak1, Enrique Chiu Han2, Cesar Eber Montelongo Hernandez3
1is a Resident Physician, Department of Psychiatry, University of Texas Southwestern Medical Center (UT Southwestern), Dallas, Texas, USA.
Journal of graduate medical education
|December 19, 2025
概括
人工智能检测工具很难准确地识别在住院个人陈述中的人工智能生成的内容. 这些工具经常错误地分类人类撰写的文本,无法可靠地区分人工智能生成的和混合内容的陈述.
科学领域:
- 医学教育 医学教育
- 人工智能的人工智能
- 学术写作 学术写作
背景情况:
- 对于在住院个人陈述中使用生成性人工智能 (AI) 存在担忧.
- 节目主管可能更喜欢人类生成的内容,促使使用AI检测工具.
- 人工智能检测个人陈述的准确性仍然不确定.
研究的目的:
- 评估AI检测工具在识别AI产生的内容在住所个人陈述中的准确性.
主要方法:
- 收集了25个由人类生成的,25个由人工智能生成的 (ChatGPT-4o) 和25个混合内容的个人陈述.
- 利用四个AI检测工具 (免费和付费) 来评估AI生成的陈述概率.
- 进行了统计分析,包括多变量方差分析 (MANOVA) 和后期测试.
主要成果:
- 人工智能检测工具显示了人类生成陈述的可变概率得分,范围高达84%.
- 在陈述类型 (P<.001) 之间发现了概率得分的显著差异.
- 在混合内容和完全由人工智能生成的陈述之间存在着显著的重叠.
结论:
- 人工智能检测工具可以不准确地将高AI概率得分分配给人类生成的内容.
- 当前的人工智能检测工具对于区分混合内容和人工智能生成的个人陈述是不可靠的.
相关概念视频
Non-equilibrium in the Cell
5.3K
An important concept in studying metabolism and energy is that of chemical equilibrium. Most chemical reactions are reversible. They can proceed in both directions, releasing energy into their environment in one direction, and absorbing it from the environment in the other direction. The same is true for the chemical reactions involved in cell metabolism, such as the breaking down and building up of proteins into and from individual amino acids, respectively. Reactants within a closed system...
5.3K
Understanding Deception
142
Deception is a pervasive aspect of human communication. Empirical studies have shown that most individuals engage in some form of deceit on a daily basis, with approximately 20% of social exchanges involving deceptive elements. Lying follows a developmental trajectory, peaking during adolescence and declining with age, possibly due to the maturation of cognitive control and social accountability.Cognitive and Social Factors in Deception DetectionDespite its prevalence, accurately detecting...
142


