Related Experiment Video
Updated: Feb 12, 2026
05:10
Drug Repurposing Hypothesis Generation Using the "RE:fine Drugs" System
Published on: December 11, 2016
10.2K
A note on Type S/M errors in hypothesis testing
Jiannan Lu1, Yixuan Qiu2, Alex Deng1
1Microsoft Corporation, Redmond, Washington, USA.
Summary
This study addresses the replication crisis by proposing new methods to control for Type S and Type M errors in hypothesis testing, offering theoretical advancements and practical demonstrations.
Area of Science:
- Statistics
- Psychological Science
Background:
- The replication and reproducibility crisis in science necessitates new statistical approaches.
- Gelman and Carlin (2014) proposed controlling for Type S/M errors over Type I/II errors in hypothesis testing.
Purpose of the Study:
- To address theoretical gaps in the methodology for controlling Type S/M errors.
- To provide a closed-form expression for expected Type M error.
- To analyze the mathematical properties of Type S and Type M errors.
Main Methods:
- Derivation of a closed-form expression for expected Type M error.
- Mathematical analysis of the properties of Type S error probability and expected Type M error.
- Numerical and empirical examples to demonstrate the utility of the derived results.
Main Results:
- A closed-form expression for expected Type M error has been derived.
- Mathematical properties, including monotonicity, of Type S and Type M errors have been studied.
- The proposed methods offer advantages demonstrated through simulations and real-world data.
Conclusions:
- The study provides theoretical advancements for controlling Type S/M errors in hypothesis testing.
- The derived results offer practical tools for researchers to improve the reliability of their findings.
- This work contributes to addressing the replication crisis by enhancing statistical rigor.
Related Concept Videos
01:14Errors In Hypothesis Tests
6.1K
When performing a hypothesis test, there are four possible outcomes depending on the actual truth (or falseness) of the null hypothesis and the decision to reject or not.
6.1K
01:11Types of Hypothesis Testing
28.1K
There are three types of hypothesis tests: right-tailed, left-tailed, and two-tailed.
When the null and alternative hypotheses are stated, it is observed that the null hypothesis is a neutral statement against which the alternative hypothesis is tested. The alternative hypothesis is a claim that instead has a certain direction. If the null hypothesis claims that p = 0.5, the alternative hypothesis would be an opposing statement to this and can be put either p > 0.5, p < 0.5, or p...
When the null and alternative hypotheses are stated, it is observed that the null hypothesis is a neutral statement against which the alternative hypothesis is tested. The alternative hypothesis is a claim that instead has a certain direction. If the null hypothesis claims that p = 0.5, the alternative hypothesis would be an opposing statement to this and can be put either p > 0.5, p < 0.5, or p...
28.1K
01:13Accuracy and Errors in Hypothesis Testing
602
Hypothesis testing is a fundamental statistical tool that begins with the assumption that the null hypothesis H0 is true. During this process, two types of errors can occur: Type I and Type II. A Type I error refers to the incorrect rejection of a true null hypothesis, while a Type II error involves the failure to reject a false null hypothesis.
In hypothesis testing, the probability of making a Type I error, denoted as α, is commonly set at 0.05. This significance level indicates a 5%...
In hypothesis testing, the probability of making a Type I error, denoted as α, is commonly set at 0.05. This significance level indicates a 5%...
602
01:16Statistical Hypothesis Testing
7.0K
Hypothesis testing is a critical statistical procedure facilitating informed, evidence-based decisions. It begins with a hypothesis, which is a tentative explanation, or a prediction about a population parameter. This hypothesis can be either a null hypothesis (H0), indicating no effect or difference, or an alternative hypothesis (Ha), suggesting an effect or difference.
Statistical significance measures the probability that an observed result occurred by chance. If this probability, known as...
Statistical significance measures the probability that an observed result occurred by chance. If this probability, known as...
7.0K
01:12Types of Errors: Detection and Minimization
11.5K
Error is the deviation of the obtained result from the true, expected value or the estimated central value. Errors are expressed in absolute or relative terms.
Absolute error in a measurement is the numerical difference from the true or central value. Relative error is the ratio between absolute error and the true or central value, expressed as a percentage.
Errors can be classified by source, magnitude, and sign. There are three types of errors: systematic, random, and gross.
Systematic or...
Absolute error in a measurement is the numerical difference from the true or central value. Relative error is the ratio between absolute error and the true or central value, expressed as a percentage.
Errors can be classified by source, magnitude, and sign. There are three types of errors: systematic, random, and gross.
Systematic or...
11.5K
01:16Statistical Inference Techniques in Hypothesis Testing: Parametric Versus Nonparametric Data
495
Statistical inference techniques, paramount in hypothesis testing, differentiate into two broad categories: parametric and nonparametric statistics.
Parametric statistics, as the name suggests, assumes that data follow a specific distribution, often a normal distribution. This assumption enables robust hypothesis testing and estimation. Parametric methods, like the Student's t-test or Goodness-of-fit test, are frequently employed in biostatistics due to their robustness. For instance,...
Parametric statistics, as the name suggests, assumes that data follow a specific distribution, often a normal distribution. This assumption enables robust hypothesis testing and estimation. Parametric methods, like the Student's t-test or Goodness-of-fit test, are frequently employed in biostatistics due to their robustness. For instance,...
495