检查由决策市场选择的在线实验的可复制性
Felix Holzmeister1, Magnus Johannesson2, Colin F Camerer3
1Department of Economics, University of Innsbruck, Innsbruck, Austria.
Nature human behaviour
|November 20, 2024
概括
决策市场有效地选择了可复制的研究,显示了在线实验的各种可复制性. 总体而言,54%的研究成功复制,与实验室实验结果相比.
科学领域:
- 社会科学研究 社会科学研究
- 心理学 心理学 心理学
- 经济学 经济学 经济学
背景情况:
- 社会科学中的可复制性危机
- 需要有效的研究选择方法来进行复制.
- 评估在线实验的可靠性 在线实验的可靠性
研究的目的:
- 评估决策市场作为选择复制研究的工具.
- 为在线社会科学实验的可复制性提供经验证据.
- 将在线实验的可复制性与实验室研究进行比较.
主要方法:
- 社会科学家 (n=162) 参加了一个决策市场.
- 在41个MTurk社会科学实验的密切复制结果上进行交易 (PNAS, 2015-2018).
- 选择了前12名,下12名和2项随机研究,根据市场价格进行复制.
主要成果:
- 在前12项研究中,复制率为83%,在下12项研究中为33% (统计学意义指标).
- 总体而言,54%的研究成功复制.
- 复制效应的大小平均为原始估计的45%;复制率在54%至62%之间.
结论:
- 决策市场显示选择复制研究的可行性.
- MTurk实验的可复制性与实验室实验相当.
- 调查结果有助于了解在线研究的可靠性.
更多相关视频
相关概念视频
Randomized Experiments
6.7K
The randomization process involves assigning study participants randomly to experimental or control groups based on their probability of being equally assigned. Randomization is meant to eliminate selection bias and balance known and unknown confounding factors so that the control group is similar to the treatment group as much as possible. A computer program and a random number generator can be used to assign participants to groups in a way that minimizes bias.
Simple randomization
Simple...
Simple randomization
Simple...
6.7K
What is an Experiment?
10.4K
An experiment is a planned activity carried out under controlled conditions. The purpose of an experiment is to investigate the relationship between two variables. When one variable causes change in another, we call the first variable the explanatory or independent variable. The affected variable is called the response or dependent variable. In a randomized experiment, the researcher manipulates values of the explanatory variable and measures the resulting changes in the response variable. The...
10.4K
Crossover Experiments
2.7K
Crossover experiments, also called the repeated-measurements design, is a study design in which all experimental units are exposed to all treatments in different periods. Crossover experiments are generally used in psychology, the pharmaceutical industry, agriculture, and medicine.
Crossover designs are performed even with smaller sample sizes since the samples can act as their controls. These are better than simple randomized trials since patients are exposed to all the treatments.
Crossover designs are performed even with smaller sample sizes since the samples can act as their controls. These are better than simple randomized trials since patients are exposed to all the treatments.
2.7K
Group Design
8.9K
The most basic experimental design involves two groups: the experimental group and the control group. The two groups are designed to be the same except for one difference— experimental manipulation. The experimental group gets the experimental manipulation—that is, the treatment or variable being tested—and the control group does not. Since experimental manipulation is the only difference between the experimental and control groups, we can be sure that any differences between...
8.9K
Data Collection by Experiments
23.9K
Data collection is a systematic method of obtaining, observing, measuring, and analyzing accurate information. An experimental study is a standard method of data collection that involves the manipulation of the samples by applying some form of treatment prior to data collection. It refers to manipulating one variable to determine its changes on another variable. The sample subjected to treatment is known as “experimental units.”
An example of the experimental method is a public...
An example of the experimental method is a public...
23.9K
Experimental Designs
11.1K
An experimental design is a systematic process that allows researchers to evaluate the relationship between dependent and independent variables. There are three widely used types of experimental design - pre-experimental design, true experimental design, and quasi-experimental design. In pre-experimental design, the researcher compares the data before and after some interventions or treatments. The true-experimental design has more than one purposefully created group, a commonly measured...
11.1K


