重新定义统计学意义对P黑客和假阳性率的影响:基于代理的模型
Ben G Fitzpatrick1,2, Dennis M Gorman3, Caitlin Trombatore1
1Department of Mathematics, Loyola Marymount University, Los Angeles, California, United States of America.
PloS one
|May 16, 2024
概括
更严格的0.005的P值值可以减少学术出版的虚假发现. 虽然研究人员仍在P-hack,但增加的努力和减少的生产力降低了公布的错误阳性率.
科学领域:
- 统计建模 统计建模
- 科学出版业的科学出版.
- 研究完整性研究完整性
背景情况:
- 越来越多的人担心P值的滥用,P黑客和选择性报告导致了不可重复的发现.
- 常规的P<0.05统计学显著性值经常受到争议.
- 一个建议的解决方案是将显著性值降低到P<0.005.5.
研究的目的:
- 调查P<0.005值对P黑客行为和公布的错误阳性率的影响.
- 用以基于进化代理的方法对研究人员行为进行建模,专注于出版率.
- 为了比较单个和多个假设测试场景的P<0.05和P<0.005值下的结果.
主要方法:
- 开发了一个基于进化代理的模型,模拟了旨在最大限度地提高出版物的研究人员.
- 模拟了三个场景:单个假设测试 (P<0.05),多个假设测试 (P<0.05),和多个假设测试 (P<0.005).
- 模型输出,包括研究人员的努力,测试的假设,出版物和假阳性率,在各种效果大小中进行了分析.
主要成果:
- 更严格的0.005的P值值显著降低了公布虚假阳性结果的比率.
- 研究人员即使在P<0.005值的情况下也继续进行P-hacking,但花费了更多的精力.
- 总体而言,研究人员的生产力下降,导致公布虚假阳性率较低.
结论:
- 将P值值降低到0.005是一个有效的策略,以减轻P黑客和减少科学文献中的虚假发现.
- 这种干预在现有的学术出版系统中相对容易实施和监控.
- 这些发现表明,调整统计值可以提高研究的可靠性和可重复性.
相关概念视频
01:50Statistical Significance
20.1K
Once data is collected from both the experimental and the control groups, a statistical analysis is conducted to find out if there are meaningful differences between the two groups. A statistical analysis determines how likely any difference found is due to chance (and thus not meaningful). In psychology, group differences are considered meaningful, or significant, if the odds that these differences occurred by chance alone are 5 percent or less. Stated another way, if we repeated this...
20.1K
01:09Decision Making: P-value Method
5.3K
The process of hypothesis testing based on the P-value method includes calculating the P- value using the sample data and interpreting it.
First, a specific claim about the population parameter is proposed. The claim is based on the research question and is stated in a simple form. Further, an opposing statement to the claim is also stated. These statements can act as null and alternative hypotheses: a null hypothesis would be a neutral statement while the alternative hypothesis can...
First, a specific claim about the population parameter is proposed. The claim is based on the research question and is stated in a simple form. Further, an opposing statement to the claim is also stated. These statements can act as null and alternative hypotheses: a null hypothesis would be a neutral statement while the alternative hypothesis can...
5.3K
01:10P-value
6.8K
P-value is one of the most crucial concepts in statistics.
P-value stands for the probability value. P-value is the probability that, if the null hypothesis is true, the results from another randomly selected sample will be as extreme or more extreme as the results obtained from the given sample.
A large P-value calculated from the data indicates to not reject the null hypothesis. But a higher P-value does not mean that the null hypothesis is true. The smaller the P-value, the more...
P-value stands for the probability value. P-value is the probability that, if the null hypothesis is true, the results from another randomly selected sample will be as extreme or more extreme as the results obtained from the given sample.
A large P-value calculated from the data indicates to not reject the null hypothesis. But a higher P-value does not mean that the null hypothesis is true. The smaller the P-value, the more...
6.8K
01:16Statistical Hypothesis Testing
1.9K
Hypothesis testing is a critical statistical procedure facilitating informed, evidence-based decisions. It begins with a hypothesis, which is a tentative explanation, or a prediction about a population parameter. This hypothesis can be either a null hypothesis (H0), indicating no effect or difference, or an alternative hypothesis (Ha), suggesting an effect or difference.
Statistical significance measures the probability that an observed result occurred by chance. If this probability, known as...
Statistical significance measures the probability that an observed result occurred by chance. If this probability, known as...
1.9K
01:13Accuracy and Errors in Hypothesis Testing
198
Hypothesis testing is a fundamental statistical tool that begins with the assumption that the null hypothesis H0 is true. During this process, two types of errors can occur: Type I and Type II. A Type I error refers to the incorrect rejection of a true null hypothesis, while a Type II error involves the failure to reject a false null hypothesis.
In hypothesis testing, the probability of making a Type I error, denoted as α, is commonly set at 0.05. This significance level indicates a 5%...
In hypothesis testing, the probability of making a Type I error, denoted as α, is commonly set at 0.05. This significance level indicates a 5%...
198
01:04Significance Testing: Overview
3.4K
Significance testing is a set of statistical methods used to test whether a claim about a parameter is valid. In analytical chemistry, significance testing is used primarily to determine whether the difference between two values comes from determinate or random errors. The effect of a particular change in the measurement protocol, analyst, or sample itself can cause a deviation from the expected result. In the case of a suspected deviation/outlier, we need to be able to confirm mathematically...
3.4K