一个随机对照试验,在同行评审讨论中对评审者相互匿名进行评审
Charvi Rastogi1, Xiangchen Song1, Zhijing Jin2
1Machine Learning Department, Carnegie Mellon University, Pittsburgh, Pennsylvania, United States of America.
PloS one
|December 27, 2024
概括
在同行评审中匿名审稿人的讨论略有改善了讨论质量和审稿人的体验. 在人工智能不确定性会议 (UAI) 上的这项研究表明,匿名性有利于科学过程.
科学领域:
- 人工智能的人工智能
- 计算机科学 计算机科学
- 科学出版科学出版
背景情况:
- 同行评审对于科学诚信至关重要.
- 评论员的讨论可能会受到非匿名互动的影响.
- 在讨论期间审查员匿名性的影响尚不清楚.
研究的目的:
- 调查审稿人的匿名性对同行评审中的讨论动态和结果的影响.
- 为了在现实世界中比较匿名与非匿名的评论员讨论.
主要方法:
- 在2022年人工智能 (UAI) 不确定性会议上进行了一项随机对照试验.
- 实施了两个条件:匿名和非匿名类型讨论论坛.
- 通过讨论日志和匿名评论员调查收集数据.
主要成果:
- 匿名讨论显示,在评估的标准中出现了小而显著的改善.
- 分析包括讨论量,资历影响,礼貌和评论员满意度.
- 审查员的经验和偏好在条件之间被系统地比较.
结论:
- 匿名审稿人讨论似乎在同行评审设置中提供了优势.
- 研究结果表明,匿名性可以积极影响科学话语的质量和公平性.
- 进一步的研究可能会在学术会议和期刊上探索更广泛的实施.
更多相关视频
08:53Integrating Computerized Linguistic and Social Network Analyses to Capture Addiction Recovery Capital in an Online Community
Published on: May 31, 2019
5.1K
06:28E-Patient Counseling Trial E-PACO: Computer Based Education versus Nurse Counseling for Patients to Prepare for Colonoscopy
Published on: August 1, 2019
8.2K
相关概念视频
01:13Randomized Experiments
6.6K
The randomization process involves assigning study participants randomly to experimental or control groups based on their probability of being equally assigned. Randomization is meant to eliminate selection bias and balance known and unknown confounding factors so that the control group is similar to the treatment group as much as possible. A computer program and a random number generator can be used to assign participants to groups in a way that minimizes bias.
Simple randomization
Simple...
Simple randomization
Simple...
6.6K
01:11Blinding
2.3K
Blinding is a commonly used method of not telling participants which treatment a subject is receiving. Blinding is a critical part of a randomized control trial or RCT. It reduces the bias that affects the results. In an RCT, blinding is used in the form of a placebo. A placebo effect occurs when untreated subjects falsely believe they have received the treatment and report improved symptoms. A placebo or a dummy treatment is administered to subjects to negate the bias caused by such an effect.
2.3K
02:07Blind Procedures
10.6K
Ideally, the people who observe and record the children’s behavior are unaware of who was assigned to the experimental or control group, in order to control for experimenter bias. Experimenter bias refers to the possibility that a researcher’s expectations might skew the results of the study. Remember, conducting an experiment requires a lot of planning, and the people involved in the research project have a vested interest in supporting their hypotheses. If the observers knew which...
10.6K
01:56Ethics in Research
22.8K
Today, scientists agree that good research is ethical in nature and is guided by a basic respect for human dignity and safety. However, this has not always been the case. Modern researchers must demonstrate that the research they perform is ethically sound.
22.8K
02:01Group Design
8.8K
The most basic experimental design involves two groups: the experimental group and the control group. The two groups are designed to be the same except for one difference— experimental manipulation. The experimental group gets the experimental manipulation—that is, the treatment or variable being tested—and the control group does not. Since experimental manipulation is the only difference between the experimental and control groups, we can be sure that any differences between...
8.8K
04:49Robbers Cave
14.2K
During the 1950s, the landmark Robbers Cave experiment demonstrated that when groups must compete with one another, intergroup conflict, hostility, and even violence may result. At the Oklahoman summer camp, two troops of boys—termed the Rattlers and the Eagles—took part in a week-long tournament. During this time, their negativity culminated in derogatory name-calling, fistfights, and even vandalism and destruction of property. However, this work also revealed that such tension...
14.2K