Related Experiment Video
Updated: Nov 9, 2025
10:46
A Method of Trigonometric Modelling of Seasonal Variation Demonstrated with Multiple Sclerosis Relapse Data
Published on: December 9, 2015
10.8K
Predicting replicability-Analysis of survey and prediction market data from large-scale forecasting projects
Michael Gordon1, Domenico Viganola2, Anna Dreber3
1Massey University, Auckland, New Zealand.
Plos One
|April 14, 2021
Summary
Scientists can predict research reproducibility using prediction markets and surveys. These methods aggregate community beliefs to forecast replication outcomes with notable accuracy, aiding science policy.
Area of Science:
- Social and behavioral sciences
- Research reproducibility
- Science policy
Background:
- The reproducibility of scientific research is a growing concern in science policy.
- Large-scale replication projects aim to assess reproducibility across academic fields.
Purpose of the Study:
- To analyze data from four studies forecasting replication outcomes in social and behavioral sciences.
- To evaluate the effectiveness of prediction markets and surveys in eliciting and aggregating expert beliefs about research reproducibility.
Main Methods:
- Pooled data from 103 published findings where expert beliefs were elicited.
- Utilized prediction markets and surveys to gather information from human experts.
- Analyzed the accuracy of prediction markets and the correlation of survey responses with replication outcomes.
Main Results:
- Prediction markets accurately forecast direct replication outcomes with 73% accuracy.
- Both prediction market prices and average survey responses significantly correlate with replication outcomes (r = 0.581 and r = 0.564, respectively).
- A significant relationship was found between the p-values of original findings and their replication success.
Conclusions:
- The scientific community possesses valuable information regarding the replicability of research findings.
- Prediction markets and surveys are effective tools for eliciting and aggregating this community knowledge.
- The findings support the use of these methods for forecasting replication outcomes and informing science policy.
Related Concept Videos
01:18Steps in Outbreak Investigation
296
In the ever-evolving field of public health, statistical analysis serves as a cornerstone for understanding and managing disease outbreaks. By leveraging various statistical tools, health professionals can predict potential outbreaks, analyze ongoing situations, and devise effective responses to mitigate impact. For that to happen, there are a few possible stages of the analysis:
296
01:18Microsoft Excel: Regression Analysis
1.2K
Regression analysis in Microsoft Excel is a powerful statistical method for examining the relationship between a dependent variable and one or more independent variables. It's used extensively in fields such as economics, biology, and business to predict outcomes, understand relationships, and make data-driven decisions. The most common type is linear regression, which attempts to fit a straight line through the data points to model the relationship between variables.
To perform regression...
To perform regression...
1.2K
02:16Surveys
16.2K
Often, psychologists develop surveys as a means of gathering data. Surveys are lists of questions to be answered by research participants, and can be delivered as paper-and-pencil questionnaires, administered electronically, or conducted verbally. Generally, the survey itself can be completed in a short time, and the ease of administering a survey makes it easy to collect data from a large number of people.
16.2K
01:03Prediction Intervals
2.6K
The interval estimate of any variable is known as the prediction interval. It helps decide if a point estimate is dependable.
However, the point estimate is most likely not the exact value of the population parameter, but close to it. After calculating point estimates, we construct interval estimates, called confidence intervals or prediction intervals. This prediction interval comprises a range of values unlike the point estimate and is a better predictor of the observed sample value, y.
However, the point estimate is most likely not the exact value of the population parameter, but close to it. After calculating point estimates, we construct interval estimates, called confidence intervals or prediction intervals. This prediction interval comprises a range of values unlike the point estimate and is a better predictor of the observed sample value, y.
2.6K
01:52Regression Toward the Mean
6.6K
Regression toward the mean (“RTM”) is a phenomenon in which extremely high or low values—for example, and individual’s blood pressure at a particular moment—appear closer to a group’s average upon remeasuring. Although this statistical peculiarity is the result of random error and chance, it has been problematic across various medical, scientific, financial and psychological applications. In particular, RTM, if not taken into account, can interfere when...
6.6K
01:25Multiple Regression
3.4K
Multiple regression assesses a linear relationship between one response or dependent variable and two or more independent variables. It has many practical applications.
Farmers can use multiple regression to determine the crop yield based on more than one factor, such as water availability, fertilizer, soil properties, etc. Here, the crop yield is the response or dependent variable as it depends on the other independent variables. The analysis requires the construction of a scatter plot...
Farmers can use multiple regression to determine the crop yield based on more than one factor, such as water availability, fertilizer, soil properties, etc. Here, the crop yield is the response or dependent variable as it depends on the other independent variables. The analysis requires the construction of a scatter plot...
3.4K