申请居留权评选委员会 识别人工智能产生的个人陈述中的歧视能力
Issam Koleilat1, Advaith Bongu2, Sumy Chang1
1Department of Surgery, Community Medical Center, RWJ/Barnabas Health, Tom's River, New Jersey.
Journal of surgical education
|April 28, 2024
概括
人类审稿人很难区分人工智能生成的个人陈述 (PS) 和人类撰写的个人陈述,这影响了住院招聘. 这种困难凸显了需要重新评估应用程序流程,因为人工智能变得越来越普遍.
科学领域:
- 医疗教育中的人工智能
- 在医疗保健应用中使用自然语言处理.
- 外科住院招生入学情况
背景情况:
- 人工智能 (AI) 算法现在可以生成类似人类的文本,这在学术和专业环境中带来了挑战.
- 个人陈述 (PS) 是外科住院申请流程的关键组成部分,提供了超越客观指标的申请人品质的洞察力.
- 人工智能文本生成日益复杂,需要对其对传统应用程序审查方法的影响进行评估.
研究的目的:
- 评估人类审稿人的能力,准确区分由人类撰写的个人陈述 (PS) 和由人工智能 (AI) 软件生成的个人陈述.
- 评估人工智能生成的文本对居留选择过程的影响,特别是关于面试报价.
主要方法:
- 作为样本,从外科住院计划档案中使用了四份由人类撰写的个人陈述 (PS).
- 两个人工智能平台产生了9个额外的PS,创建了一个用于评估的数据集.
- 四位经验丰富的外科医生在住院选择委员会中担任盲目审查员,以确定作者身份,人工智能也评估作者身份.
主要成果:
- 人类审查者显示灵敏度为0.87,但特异性较低,为0.37,在识别人工智能产生的PS时,整体准确度为0.55.
- 评审者之间的评审者之间的可靠性很差 (kappa = 0.067),这表明在作者归因方面存在重大分歧.
- 感知到的人类作者显著增加了接受采访报价的几率 (OR=7,p=0.0144),强调了感知到作者带来的偏见.
结论:
- 对专家审稿人来说,区分人工智能生成的个人陈述和人类撰写的个人陈述是极具挑战性的.
- 人工智能在生成申请材料中的普遍使用可能需要对居民招聘和选择过程进行根本性审查和潜在的重组.
- 随着客观申请人数据变得越来越稀缺,在招生中依赖和解释个人陈述需要在AI时代仔细考虑.
更多相关视频
06:02Evaluating Usability Aspects of a Mixed Reality Solution for Immersive Analytics in Industry 4.0 Scenarios
Published on: October 6, 2020
2.2K
07:34Perceptual and Category Processing of the Uncanny Valley Hypothesis' Dimension of Human Likeness: Some Methodological Issues
Published on: June 3, 2013
17.3K
相关概念视频
Stereotypes, Prejudice, and Discrimination
90.2K
Humans are very diverse and although we share many similarities, we also have many differences. The social groups we belong to help form our identities (Tajfel, 1974). These differences may be difficult for some people to reconcile, which may lead to prejudice toward people who are different. Prejudice is a negative attitude and feeling toward an individual based solely on one’s membership in a particular social group (Allport, 1954; Brown, 2010). Prejudice is common against people who...
90.2K
Confirmation Biases
5.5K
The confirmation bias is the tendency to focus on information that confirms our existing beliefs and ignore information that is inconsistent with our expectations. For example, if you think that your professor is not very nice, you notice all of the instances of rude behavior exhibited by the professor while ignoring the countless pleasant interactions he is involved in on a daily basis. Have you ever fallen prey to the confirmation bias, either as the source or target of such bias?
5.5K
Surveys
14.8K
Often, psychologists develop surveys as a means of gathering data. Surveys are lists of questions to be answered by research participants, and can be delivered as paper-and-pencil questionnaires, administered electronically, or conducted verbally. Generally, the survey itself can be completed in a short time, and the ease of administering a survey makes it easy to collect data from a large number of people.
14.8K
Stereotype Content Model
14.7K
The Stereotype Content Model (SCM) was first proposed by Susan Fiske and her colleagues (Fiske, Cuddy, Glick & Xu, 2002; see also Fiske, 2012 and Fiske, 2017). The SCM specifies that when someone encounters a new group, they will stereotype them based on two metrics: warmth—or that group’s perceived intent, and how likely they are to provide help or inflict harm—and competence—or their ability to carry out that objective. Depending on the warmth-competence...
14.7K
The Representativeness Heuristic
15.8K
The representative heuristic describes a biased way of thinking, in which you unintentionally stereotype someone or something. For example, you may assume that your professors spend their free time reading books and engaging in intellectual conversation, because the idea of them spending their time playing volleyball or visiting an amusement park does not fit in with your stereotypes of professors.
15.8K
Stereotype Threat and Self-fulfilling Prophecies
37.6K
When we hold a stereotype about a person, we have expectations that he or she will fulfill that stereotype. A self-fulfilling prophecy is an expectation held by a person that alters his or her behavior in a way that tends to make it true. When we hold stereotypes about a person, we tend to treat the person according to our expectations. This treatment can influence the person to act according to our stereotypic expectations, thus confirming our stereotypic beliefs. Research by Rosenthal and...
37.6K
