Related Experiment Video
Updated: Jun 27, 2025
03:36
Development of Compendium for Esophageal Squamous Cell Carcinoma
Published on: April 12, 2024
407
From significance testing to estimation and Open Science: How esci can help
Robert Calin-Jageman1, Geoff Cumming2
1Department of Psychology, Dominican University, River Forest, IL, USA.
Summary
Researchers should shift from null hypothesis significance testing to estimation and embrace Open Science. This involves using effect sizes and confidence intervals, supported by new tools like esci (estimation statistics with confidence intervals).
Area of Science:
- Statistics
- Research Methodology
- Open Science
Background:
- Traditional null hypothesis significance testing (NHST) and p-values have demonstrated unreliability in research.
- There is a growing movement towards Open Science practices to enhance research integrity and replicability.
- Estimation and meta-analysis, termed "the new statistics," offer advantages over traditional testing methods.
Purpose of the Study:
- To advocate for a shift from NHST to estimation and the adoption of Open Science practices.
- To introduce and describe esci (estimation statistics with confidence intervals), a software tool designed to support estimation.
- To highlight the benefits of estimation for improving statistical understanding and research practices.
Main Methods:
- Critique of null hypothesis significance testing and the unreliability of p-values.
- Explanation of the advantages of estimation and meta-analysis.
- Description of the esci software, including online simulations, an R package, and integration with jamovi and JASP.
Main Results:
- The study outlines flaws in NHST and demonstrates the unreliability of p-values.
- esci provides tools for calculating effect sizes and confidence intervals, creating visualizations, and conducting hypothesis evaluation with interval nulls.
- The software supports understanding of statistical concepts through online activities.
Conclusions:
- Researchers are encouraged to adopt estimation and Open Science practices for more robust and transparent research.
- esci is presented as a valuable tool for students and researchers transitioning to estimation-based statistics.
- The adoption of "the new statistics" and associated tools like esci can improve the quality and replicability of scientific findings.
Related Concept Videos
01:04Significance Testing: Overview
3.4K
Significance testing is a set of statistical methods used to test whether a claim about a parameter is valid. In analytical chemistry, significance testing is used primarily to determine whether the difference between two values comes from determinate or random errors. The effect of a particular change in the measurement protocol, analyst, or sample itself can cause a deviation from the expected result. In the case of a suspected deviation/outlier, we need to be able to confirm mathematically...
3.4K
01:50Statistical Significance
20.1K
Once data is collected from both the experimental and the control groups, a statistical analysis is conducted to find out if there are meaningful differences between the two groups. A statistical analysis determines how likely any difference found is due to chance (and thus not meaningful). In psychology, group differences are considered meaningful, or significant, if the odds that these differences occurred by chance alone are 5 percent or less. Stated another way, if we repeated this...
20.1K
01:12Statistical Software for Data Analysis and Clinical Trials
540
Statistical software is pivotal in data analysis and clinical trials by providing tools to analyze data, draw conclusions, and make predictions. These software packages range from simple data management applications to complex analytical platforms, supporting various statistical tests, models, and simulation techniques. Their significance lies in their ability to handle vast amounts of data with precision and efficiency, enabling researchers to validate hypotheses, identify trends, and make...
540
01:14Decision Making: Traditional Method
4.0K
The process of hypothesis testing based on the traditional method includes calculating the critical value, testing the value of the test statistic using the sample data, and interpreting these values.
First, a specific claim about the population parameter is decided based on the research question and is stated in a simple form. Further, an opposing statement to this claim is also stated. These statements can act as null and alternative hypotheses, out of which a null hypothesis would be a...
First, a specific claim about the population parameter is decided based on the research question and is stated in a simple form. Further, an opposing statement to this claim is also stated. These statements can act as null and alternative hypotheses, out of which a null hypothesis would be a...
4.0K
01:16Statistical Hypothesis Testing
1.9K
Hypothesis testing is a critical statistical procedure facilitating informed, evidence-based decisions. It begins with a hypothesis, which is a tentative explanation, or a prediction about a population parameter. This hypothesis can be either a null hypothesis (H0), indicating no effect or difference, or an alternative hypothesis (Ha), suggesting an effect or difference.
Statistical significance measures the probability that an observed result occurred by chance. If this probability, known as...
Statistical significance measures the probability that an observed result occurred by chance. If this probability, known as...
1.9K
01:10Introduction to the Sign Test
816
The sign test is an important tool in nonparametric statistics, offering a straightforward yet effective method for analyzing matched pairs, nominal data, or hypotheses concerning the median of a population. It transforms data points into positive or negative signs, avoiding the need for assumptions about data distribution and instead focusing on the direction of change. It is particularly valuable when data does not conform to the normal distribution requirements of many parametric tests. For...
816