人类还是机器? 在个人陈述中对人工智能生成的书面检测进行比较分析
Margaret A Goodman1,2,3,4, Anthony M Lee1,2,3,4, Zachary Schreck1,2,3,4
1Margaret A. Goodman, Program in Physical Therapy in the Mayo Clinic School of Health Sciences at the Mayo Clinic College of Medicine and Science and in the Department of Physical Medicine and Rehabilitation at the Mayo Clinic.
Journal, physical therapy education
|January 14, 2025
概括
人类读者,反复量化分析 (RQA) 和GPTZero准确地区分人工智能生成的人类书面个人陈述. 这些方法为在招生过程中保持学术诚信提供了有价值的工具.
科学领域:
- 医学教育 医学教育
- 健康 专业 教育 卫生 专业 教育
- 学术诚信 学术诚信
背景情况:
- 先进的人工智能的兴起,包括ChatGPT和Google Gemini,对学术提交的真实性提出了挑战.
- 以前的研究表明,在检测人工智能生成的文本方面,成功率可变,需要进一步调查.
研究的目的:
- 为了评估人类读者的有效性,Recurrence Quantification Analysis (RQA) 和GPTZero AI检测工具在区分人工智能产生的与人类写的个人陈述.
- 评估这些检测方法对物理治疗师教育计划录取的影响.
主要方法:
- 分析了50份人类撰写的个人陈述和50份人工智能生成的个人陈述.
- 分析是由两个人类评分员进行的,RQA用于词汇复杂性,GPTZero用于AI特定的文本特征.
主要成果:
- 人类评分器获得了高的一致性 (κ = 0.92) 和准确性 (97%-99%).
- RQA参数 (重复性,最大线) 和GPTZero参数 (简单性,困难性,可读性) 有效地区分了人类和人工智能生成的文本,ROC曲线下的面积> 0.768.8.
结论:
- 人类读者,RQA和GPTZero在区分人工智能产生的个人陈述和人类撰写的个人陈述方面表现出不同但显著的准确性.
- 研究结果强调了这些方法对学术录取的重要性,并建议未来研究综合方法.
- 为申请人,管理机构和机构提出战略,以维护招生过程中的诚信.
相关概念视频
03:00Self-Evaluation: Self-Enhancement and Self-Verification
5.2K
Social psychologists have documented that feeling good about ourselves and maintaining positive self-esteem is a powerful motivator of human behavior (Tavris & Aronson, 2008). In the United States, members of the predominant culture typically think very highly of themselves and view themselves as good people who are above average on many desirable traits (Ehrlinger, Gilovich, & Ross, 2005). Often, our behavior, attitudes, and beliefs are affected when we experience a threat to our...
5.2K
01:29Reliability and Validity
12.7K
Reliability and validity are two important considerations that must be made with any type of data collection. Reliability refers to the ability to consistently produce a given result. In the context of psychological research, this would mean that any instruments or tools used to collect data do so in consistent, reproducible ways.
12.7K
02:05Self-Presentation: Self-Monitoring and Self-Handicapping
38.8K
People can go to great lengths to protect their self-image and present themselves in ways that they want others to see them. Sociologist Erving Goffman presented the idea that a person is like an actor on a stage. Calling his theory dramaturgy, Goffman believed that we use “impression management” to present ourselves to others as we hope to be perceived. Each situation is a new scene, and individuals perform different roles depending on who is present (Goffman, 1959). Think about...
38.8K
02:45Self-Discrepancy Theory
18.3K
One influential perspective on what motivates people's behavior is detailed in Tory Higgin's self-discrepancy theory (Higgins, 1987). He proposed that people hold disagreeing internal representations of themselves that lead to different emotional states.
18.3K
02:16Stereotype Content Model
14.0K
The Stereotype Content Model (SCM) was first proposed by Susan Fiske and her colleagues (Fiske, Cuddy, Glick & Xu, 2002; see also Fiske, 2012 and Fiske, 2017). The SCM specifies that when someone encounters a new group, they will stereotype them based on two metrics: warmth—or that group’s perceived intent, and how likely they are to provide help or inflict harm—and competence—or their ability to carry out that objective. Depending on the warmth-competence...
14.0K
02:09Stereotype Threat and Self-fulfilling Prophecies
37.4K
When we hold a stereotype about a person, we have expectations that he or she will fulfill that stereotype. A self-fulfilling prophecy is an expectation held by a person that alters his or her behavior in a way that tends to make it true. When we hold stereotypes about a person, we tend to treat the person according to our expectations. This treatment can influence the person to act according to our stereotypic expectations, thus confirming our stereotypic beliefs. Research by Rosenthal and...
37.4K