Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Sample Size Calculation01:19

Sample Size Calculation

3.9K
Knowledge of the sample size is the first requirement to conduct random sampling or an experiment. The sample size is the total number of units, observations, or groups (in some cases) used to get the data to estimate a population parameter. As the name suggests, the sample size is that of the sample drawn from the population and differs from the population size.
The sample size for the given experiment or sampling effort is fundamental to any study design. Sample size decides the number of...
3.9K
Accuracy and Errors in Hypothesis Testing01:13

Accuracy and Errors in Hypothesis Testing

347
Hypothesis testing is a fundamental statistical tool that begins with the assumption that the null hypothesis H0 is true. During this process, two types of errors can occur: Type I and Type II. A Type I error refers to the incorrect rejection of a true null hypothesis, while a Type II error involves the failure to reject a false null hypothesis.
In hypothesis testing, the probability of making a Type I error, denoted as α, is commonly set at 0.05. This significance level indicates a 5%...
347
Types of Biopharmaceutical Studies: Controlled and Non-Controlled Approaches01:23

Types of Biopharmaceutical Studies: Controlled and Non-Controlled Approaches

186
Biopharmaceutical studies constitute a vital field aiming to enhance drug delivery methods and refine therapeutic approaches, drawing upon diverse interdisciplinary knowledge. In research methodologies, the choice between controlled and non-controlled studies significantly influences the study's reliability and accuracy.
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast,...
186
Errors In Hypothesis Tests01:14

Errors In Hypothesis Tests

4.7K
When performing a hypothesis test, there are four possible outcomes depending on the actual truth (or falseness) of the null hypothesis and the decision to reject or not.
4.7K
Sample Proportion and Population Proportion01:20

Sample Proportion and Population Proportion

5.7K
Collecting samples or responses from an entire population takes significant time and effort, so a researcher collects responses from only a sample of that population. Suppose a study needs to collect information about a specific mobile application. After sample collection, the researcher analyzes the data and discovers that most individuals in the sample use that specific mobile application. The sample proportion measures the number of individuals in a sample who either use or don't use the...
5.7K
Sampling Distribution01:12

Sampling Distribution

13.7K
Given simple random samples of size n from a given population with a measured characteristic such as mean, proportion, or standard deviation for each sample, the probability distribution of all the measured characteristics is called a sampling distribution. How much the statistic varies from one sample to another is known as the sampling variability of a statistic. You typically measure the sampling variability of a statistic by its standard error. The standard error of the mean is an example...
13.7K

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Highlighting Rigor, Reproducibility, and Transparency of Research in an Academic CV.

Clinical and translational science·2026
Same author

Longitudinal Extension of the Win Odds for Ordinal Repeated Measurements.

Statistics in medicine·2026
Same author

Results of Dynamic Decompression of the Lateral Femoral Cutaneous Nerve in Idiopathic Meralgia Paresthetica: A Case Series of 109 Procedures.

Neurosurgery·2026
Same author

An Empirical Assessment of the Cost of Dichotomization of the Outcome of Clinical Trials.

Statistics in medicine·2026
Same author

Registration and reporting of clinical trials affiliated with California universities and with primary completion date from 2014 to 2017.

Trials·2025
Same author

Meta-analysis with a single study.

Statistical methods in medical research·2025

Related Experiment Video

Updated: Sep 27, 2025

Problem-Solving Before Instruction PS-I: A Protocol for Assessment and Intervention in Students with Different Abilities
10:26

Problem-Solving Before Instruction PS-I: A Protocol for Assessment and Intervention in Students with Different Abilities

Published on: September 11, 2021

4.1K

How large should the next study be? Predictive power and sample size requirements for replication studies.

Erik W van Zwet1, Steven N Goodman2,3,4

  • 1Department of Biomedical Data Sciences, Leiden University Medical Center, Leiden, The Netherlands.

Statistics in Medicine
|April 9, 2022
PubMed
Summary

Replicating a statistically significant finding requires larger sample sizes. Even with a p-value of 0.05, replication success is below 30%, highlighting the importance of robust experimental design and statistical power.

Keywords:
Cochrane Reviewactual powerclinical trialpredictive powertype S error

More Related Videos

Novel Object Recognition and Object Location Behavioral Testing in Mice on a Budget
05:57

Novel Object Recognition and Object Location Behavioral Testing in Mice on a Budget

Published on: November 20, 2018

56.8K
A Cost Effective and Adaptable Scratch Migration Assay
08:59

A Cost Effective and Adaptable Scratch Migration Assay

Published on: June 30, 2020

5.6K

Related Experiment Videos

Last Updated: Sep 27, 2025

Problem-Solving Before Instruction PS-I: A Protocol for Assessment and Intervention in Students with Different Abilities
10:26

Problem-Solving Before Instruction PS-I: A Protocol for Assessment and Intervention in Students with Different Abilities

Published on: September 11, 2021

4.1K
Novel Object Recognition and Object Location Behavioral Testing in Mice on a Budget
05:57

Novel Object Recognition and Object Location Behavioral Testing in Mice on a Budget

Published on: November 20, 2018

56.8K
A Cost Effective and Adaptable Scratch Migration Assay
08:59

A Cost Effective and Adaptable Scratch Migration Assay

Published on: June 30, 2020

5.6K

Area of Science:

  • Biostatistics
  • Research Methodology
  • Scientific Reproducibility

Background:

  • The Cochrane Collaboration database of systematic reviews (CDSR) contains over 40,000 trials.
  • Assessing the probability of experimental replication is crucial for scientific validity.

Purpose of the Study:

  • To compute the replication probability (predictive power) of experiments based on their observed p-values.
  • To determine the probability of correctly estimating the direction of an effect.
  • To calculate the necessary sample size for replication studies.

Main Methods:

  • Utilized data from over 40,000 trials in the CDSR.
  • Calculated replication probability conditional on observed p-values.
  • Computed the probability of correct effect direction (related to Type S error).
  • Determined required sample sizes for replication studies to achieve specific statistical power.

Main Results:

  • Replication of a marginally significant result (p < 0.05) has <30% chance of significance.
  • Replication of a result with p < 0.01 has only a 50% chance of significance.
  • An effect with p < 0.05 has a 93% probability of correct direction; p < 0.001 yields 99% probability.
  • Replication requires >16x sample size for p < 0.05 and 3.5x for p < 0.01 to achieve 80% power.

Conclusions:

  • Failure to replicate statistical significance does not automatically imply the original finding was erroneous.
  • P-values alone are insufficient indicators of replication success.
  • Adequate sample size is critical for ensuring the reproducibility of research findings.