人工智能能骗住院选择委员会吗? 对真实申请人和生成AI的个人陈述的分析,这是一个随机的,单盲的多中心研究
Zachary C Lum1,2, Lohitha Guntupalli1, Augustine M Saiz2
1Department of Surgery, Kiran Patel School of Osteopathic and Allopathic Medicine, Nova Southeastern University, Davie, Florida.
JB & JS open access
|October 25, 2024
概括
像ChatGPT和Google BARD这样的生成人工智能 (AI) 工具可以为居住申请创建令人信服的个人陈述. 然而,教师评论员可以区分真实的陈述和人工智能生成的陈述,而ChatGPT更具欺骗性.
科学领域:
- 医学教育 医学教育
- 医疗保健中的人工智能
- 住院招生 住院招生
背景情况:
- 生成型人工智能 (AI) 能力正在迅速发展.
- 使用人工智能为医疗住院申请制作个人陈述是一个新兴的领域.
- 了解人工智能生成的个人陈述的可检测性和质量对于招生委员会至关重要.
研究的目的:
- 评估生成性AI工具 (ChatGPT和Google BARD) 创建医疗住院申请个人陈述的能力.
- 评估是否在住院选择委员会上的教师可以区分真实和人工智能生成的陈述.
- 确定个人陈述的特定指标是否可识别为人工智能生成的或真实的.
主要方法:
- 使用ChatGPT生成了15个独特的个人陈述,使用Google BARD生成了15个独特的个人陈述,基于15个真实陈述.
- 向教师审稿人提交了总共45个随机和盲目的陈述 (15个真实,15个ChatGPT,15个BARD).
- 教师评估了14个指标的陈述,包括将它们确定为真实或人工智能生成的.
主要成果:
- 教师正确识别了88%的真实陈述和90%的BARD生成的陈述,但只有44%的ChatGPT生成的陈述.
- 在识别AI与真实陈述的总体准确性为BARD的89%,包括ChatGPT时为74%.
- 在所有指标上,BARD生成的语句表现明显低于真实语句和ChatGPT语句 (p < 0.001).
- 真实陈述在个人利益,选择居留的原因,职业目标,令人信服的性质和原创性方面优于ChatGPT陈述 (p < 0.001).
结论:
- 教师可以准确地识别真实和BARD生成的个人陈述,但ChatGPT成功欺骗了审查员的56%时间.
- 虽然人工智能可以产生令人信服的陈述,但复制人文元素,如个人细微差别和个人经验仍然具有挑战性.
- 在评估申请时,住院选择委员会应该考虑强调反映真实个人经验的指标,例如兴趣,动机和独创性.
相关概念视频
Reliability and Validity
12.7K
Reliability and validity are two important considerations that must be made with any type of data collection. Reliability refers to the ability to consistently produce a given result. In the context of psychological research, this would mean that any instruments or tools used to collect data do so in consistent, reproducible ways.
12.7K
Stereotype Threat and Self-fulfilling Prophecies
37.5K
When we hold a stereotype about a person, we have expectations that he or she will fulfill that stereotype. A self-fulfilling prophecy is an expectation held by a person that alters his or her behavior in a way that tends to make it true. When we hold stereotypes about a person, we tend to treat the person according to our expectations. This treatment can influence the person to act according to our stereotypic expectations, thus confirming our stereotypic beliefs. Research by Rosenthal and...
37.5K
The Representativeness Heuristic
15.8K
The representative heuristic describes a biased way of thinking, in which you unintentionally stereotype someone or something. For example, you may assume that your professors spend their free time reading books and engaging in intellectual conversation, because the idea of them spending their time playing volleyball or visiting an amusement park does not fit in with your stereotypes of professors.
15.8K
Cause and Effect
10.9K
While variables are sometimes correlated because one does cause the other, it could also be that some other factor, a confounding variable, is actually causing the systematic movement in our variables of interest. For instance, as sales in ice cream increase, so does the overall rate of crime. Is it possible that indulging in your favorite flavor of ice cream could send you on a crime spree? Or, after committing crime do you think you might decide to treat yourself to a cone?
10.9K
Hypothesis: Accept or Fail to Reject?
27.6K
The outcome of any hypothesis testing leads to rejecting or not rejecting the null hypothesis. This decision is taken based on the analysis of the data, an appropriate test statistic, an appropriate confidence level, the critical values, and P-values. However, when the evidence suggests that the null hypothesis cannot be rejected, is it right to say, 'Accept' the null hypothesis?
There are two ways to indicate that the null hypothesis is not rejected. 'Accept' the null...
There are two ways to indicate that the null hypothesis is not rejected. 'Accept' the null...
27.6K
Confirmation Biases
5.5K
The confirmation bias is the tendency to focus on information that confirms our existing beliefs and ignore information that is inconsistent with our expectations. For example, if you think that your professor is not very nice, you notice all of the instances of rude behavior exhibited by the professor while ignoring the countless pleasant interactions he is involved in on a daily basis. Have you ever fallen prey to the confirmation bias, either as the source or target of such bias?
5.5K


