Related Experiment Video
Updated: Sep 23, 2025
14:01
Making Record-efficiency SnS Solar Cells by Thermal Evaporation and Atomic Layer Deposition
Published on: May 22, 2015
42.9K
Statistical significance and its critics: practicing damaging science, or damaging scientific practice?
1Virginia Tech, Blacksburg, USA.
Synthese
|May 17, 2022
Summary
Statistical significance testing and p-values are crucial for scientific claims. Misinterpreting these tools, not the tools themselves, causes issues, and abandoning them risks scientific integrity.
Area of Science:
- Statistics in scientific research
- Methodology in empirical studies
Background:
- The replication crisis has intensified debate surrounding statistical significance testing and p-values.
- Concerns exist that these statistical methods are detrimental to scientific practice, leading some to advocate for their abandonment.
Purpose of the Study:
- To address the controversy surrounding statistical significance testing and p-values.
- To argue that criticisms stem from misuse and misunderstanding, and proposed remedies may harm science.
- To defend the utility of statistical significance tests in validating scientific findings.
Main Methods:
- Explanation of competing statistical philosophies.
- Reinterpretation of statistical significance tests to address common misuses.
- Argumentation against abandoning p-value thresholds.
Main Results:
- Criticisms of statistical significance testing arise from user error, not inherent flaws.
- Abandoning p-value thresholds can worsen data dredging and selection bias.
- Statistical significance tests are essential for distinguishing genuine patterns from random variability.
Conclusions:
- Statistical significance testing and p-values remain vital tools when correctly applied.
- Proposed alternatives to statistical significance testing may undermine the scientific process.
- Proper understanding and application of statistical methods are key to robust scientific inquiry.
Related Concept Videos
01:04Significance Testing: Overview
3.9K
Significance testing is a set of statistical methods used to test whether a claim about a parameter is valid. In analytical chemistry, significance testing is used primarily to determine whether the difference between two values comes from determinate or random errors. The effect of a particular change in the measurement protocol, analyst, or sample itself can cause a deviation from the expected result. In the case of a suspected deviation/outlier, we need to be able to confirm mathematically...
3.9K
01:50Statistical Significance
20.4K
Once data is collected from both the experimental and the control groups, a statistical analysis is conducted to find out if there are meaningful differences between the two groups. A statistical analysis determines how likely any difference found is due to chance (and thus not meaningful). In psychology, group differences are considered meaningful, or significant, if the odds that these differences occurred by chance alone are 5 percent or less. Stated another way, if we repeated this...
20.4K
01:16Statistical Hypothesis Testing
2.1K
Hypothesis testing is a critical statistical procedure facilitating informed, evidence-based decisions. It begins with a hypothesis, which is a tentative explanation, or a prediction about a population parameter. This hypothesis can be either a null hypothesis (H0), indicating no effect or difference, or an alternative hypothesis (Ha), suggesting an effect or difference.
Statistical significance measures the probability that an observed result occurred by chance. If this probability, known as...
Statistical significance measures the probability that an observed result occurred by chance. If this probability, known as...
2.1K
01:16Critical Region, Critical Values and Significance Level
12.2K
The critical region, critical value, and significance level are interdependent concepts crucial in hypothesis testing.
In hypothesis testing, a sample statistic is converted to a test statistic using z, t, or chi-square distribution. A critical region is an area under the curve in probability distributions demarcated by the critical value. When the test statistic falls in this region, it suggests that the null hypothesis must be rejected. As this region contains all those values of the...
In hypothesis testing, a sample statistic is converted to a test statistic using z, t, or chi-square distribution. A critical region is an area under the curve in probability distributions demarcated by the critical value. When the test statistic falls in this region, it suggests that the null hypothesis must be rejected. As this region contains all those values of the...
12.2K
01:14Decision Making: Traditional Method
4.2K
The process of hypothesis testing based on the traditional method includes calculating the critical value, testing the value of the test statistic using the sample data, and interpreting these values.
First, a specific claim about the population parameter is decided based on the research question and is stated in a simple form. Further, an opposing statement to this claim is also stated. These statements can act as null and alternative hypotheses, out of which a null hypothesis would be a...
First, a specific claim about the population parameter is decided based on the research question and is stated in a simple form. Further, an opposing statement to this claim is also stated. These statements can act as null and alternative hypotheses, out of which a null hypothesis would be a...
4.2K
01:13Accuracy and Errors in Hypothesis Testing
337
Hypothesis testing is a fundamental statistical tool that begins with the assumption that the null hypothesis H0 is true. During this process, two types of errors can occur: Type I and Type II. A Type I error refers to the incorrect rejection of a true null hypothesis, while a Type II error involves the failure to reject a false null hypothesis.
In hypothesis testing, the probability of making a Type I error, denoted as α, is commonly set at 0.05. This significance level indicates a 5%...
In hypothesis testing, the probability of making a Type I error, denoted as α, is commonly set at 0.05. This significance level indicates a 5%...
337