关联审查员的经验与歧视人类写的与ChatGPT写的摘要
Gabriel Levin1, Rene Pareja2, David Viveros-Carreño3,4
1Division of Gynecologic Oncology, Jewish General Hospital, McGill University, Montreal, Quebec, Canada Gabriel.levin2@mail.mcgill.ca.
概括
审查人员无法可靠地区分人类和ChatGPT撰写的摘要,只有46.3%的AI生成的摘要被正确识别. 评论员的经验和出版记录显著提高了检测率.
科学领域:
- 医学写作 医学写作
- 研究中的人工智能.
- 科学出版科学出版
背景情况:
- 像ChatGPT这样的人工智能 (AI) 工具的扩散在科学出版中带来了新的挑战.
- 区分人类撰写的和人工智能生成的科学摘要对于保持学术完整性至关重要.
- 对于研究人员和编辑来说,了解影响检测人工智能生成内容的因素是必不可少的.
研究的目的:
- 评估不同经验水平的医学审稿人的能力,以区分人类撰写和ChatGPT生成的摘要.
- 识别与精确识别人工智能生成摘要相关的因素,例如审稿人的经验和对人工智能的熟悉程度.
主要方法:
- 30名审稿人 (10名高级,10名初级,10名居民) 评估了20篇摘要 (10篇由人类撰写,10篇由ChatGPT撰写).
- 一个基于网络的调查问卷收集了人口统计数据和审查员的评估.
- 使用统计分析,包括多变量回归,以确定正确识别的预测因素.
主要成果:
- 总的来说,审稿人正确识别了50%的摘要,人写的摘要率略高 (53.7%) 与ChatGPT写的摘要率略高 (46.3%) (p=0.07).
- 高级审查员的正确识别率明显高 (60%),相比较于初级审查员和居民 (45%).
- 评论员的经验 (p=0.019) 和对AI的熟悉程度 (p=0.035) 独立地与准确的识别相关,出版物数量 (r=0.61,p<0.001) 也是如此.
结论:
- 审稿人对可靠地检测ChatGPT生成的摘要的能力有限.
- 增加评论员的经验,出版记录和对AI的熟悉程度,提高了识别AI产生的内容的准确性.
- 这些发现凸显了需要改进的策略和工具来解决AI生成的文本在科学沟通.
更多相关视频
09:20Cloud-Based Phrase Mining and Analysis of User-Defined Phrase-Category Association in Biomedical Publications
Published on: February 23, 2019
8.7K
07:34Perceptual and Category Processing of the Uncanny Valley Hypothesis' Dimension of Human Likeness: Some Methodological Issues
Published on: June 3, 2013
17.3K
相关概念视频
02:01Group Design
8.9K
The most basic experimental design involves two groups: the experimental group and the control group. The two groups are designed to be the same except for one difference— experimental manipulation. The experimental group gets the experimental manipulation—that is, the treatment or variable being tested—and the control group does not. Since experimental manipulation is the only difference between the experimental and control groups, we can be sure that any differences between...
8.9K
02:07Improving Translational Accuracy
2.6K
2.6K
02:16Surveys
14.8K
Often, psychologists develop surveys as a means of gathering data. Surveys are lists of questions to be answered by research participants, and can be delivered as paper-and-pencil questionnaires, administered electronically, or conducted verbally. Generally, the survey itself can be completed in a short time, and the ease of administering a survey makes it easy to collect data from a large number of people.
14.8K
02:16Stereotype Content Model
14.7K
The Stereotype Content Model (SCM) was first proposed by Susan Fiske and her colleagues (Fiske, Cuddy, Glick & Xu, 2002; see also Fiske, 2012 and Fiske, 2017). The SCM specifies that when someone encounters a new group, they will stereotype them based on two metrics: warmth—or that group’s perceived intent, and how likely they are to provide help or inflict harm—and competence—or their ability to carry out that objective. Depending on the warmth-competence...
14.7K
02:07Blind Procedures
10.6K
Ideally, the people who observe and record the children’s behavior are unaware of who was assigned to the experimental or control group, in order to control for experimenter bias. Experimenter bias refers to the possibility that a researcher’s expectations might skew the results of the study. Remember, conducting an experiment requires a lot of planning, and the people involved in the research project have a vested interest in supporting their hypotheses. If the observers knew which...
10.6K
02:55Stereotypes, Prejudice, and Discrimination
90.2K
Humans are very diverse and although we share many similarities, we also have many differences. The social groups we belong to help form our identities (Tajfel, 1974). These differences may be difficult for some people to reconcile, which may lead to prejudice toward people who are different. Prejudice is a negative attitude and feeling toward an individual based solely on one’s membership in a particular social group (Allport, 1954; Brown, 2010). Prejudice is common against people who...
90.2K