Related Experiment Video
Updated: Jun 27, 2026
An Efficient Protocol to Assess ERK Activity Modulation in Early Zebrafish Noonan Syndrome Models via Live FRET Microscopy and Immunofluorescence
Published on: May 2, 2025
A small p-value from an observed data is not evidence of adequate power for future similar-sized studies: a
Eshetu G Atenafu1, Jemila S Hamid, Derek Stephens
1Child Health Evaluative Sciences, The Hospital for Sick Children, Toronto, Ontario, Canada.
Background:
p-values are ubiquitous in medical research, but are often misunderstood. In addition to being misused or perhaps even abused at post-statistical analysis stage of making scientific inference and interpretations, p-values can also be a source of confusion at the design stage.
Methods:
Application of standard test statistic on observed data may result in a small p-value which in turn may give the impression that a new study that has the same sample size as the observed data, perhaps even smaller, would have adequate power. We used re-sampling method and computed statistical power to illustrate the fallacy of this conclusion. We have also calculated power using analytical formulae.
Results:
We analyzed data consisting of two group comparisons with binary as well as continuous outcome variables. For the binary outcome, the event rates for the outcome of interest in the illustrative data were 15/43 (35%) and 22/34 (65%), respectively (p-value=0.0093). Using these data, a bootstrap-based empirical power was estimated to be 75.4%. One random sample with only two-third of the original data had a p-value of 0.0066, but only an empirical power of 57.4%. Similar results were observed for a continuous outcome.
Conclusion:
Our results show that the number of zeros after the decimal point in a p-value from an observed sample cannot and should not be used to gauge the adequacy of sample size for a future study that is expected to have sufficient power to detect an effect as big as the observed.
Related Concept Videos
01:10P-value
P-value stands for the probability value. P-value is the probability that, if the null hypothesis is true, the results from another randomly selected sample will be as extreme or more extreme as the results obtained from the given sample.
A large P-value calculated from the data indicates to not reject the null hypothesis. But a higher P-value does not mean that the null hypothesis is true. The smaller the P-value, the more unlikely...
01:09Decision Making: P-value Method
First, a specific claim about the population parameter is proposed. The claim is based on the research question and is stated in a simple form. Further, an opposing statement to the claim is also stated. These statements can act as null and alternative hypotheses: a null hypothesis would be a neutral statement while the alternative hypothesis can have a...
01:11Testing a Claim about Mean: Known Population SD
Estimating a population mean requires the samples to be distributed normally. The data should be collected from the randomly selected samples having no sampling bias. The sample size needed to be higher than 30, and most importantly, the population standard deviation should be already known.
In most realistic situations, the population standard deviation is often unknown, but in rare circumstances, when it...
01:24Testing a Claim about Population Proportion
There are two methods of testing a claim about a population proportion: (1) Using the sample proportion from the data where a binomial distribution is approximated to the normal distribution and (2) Using the binomial probabilities calculated from the data.
The first method uses normal distribution as an approximation to the binomial distribution. The requirements are as follows: sample size is large...
01:16Bioequivalence Data: Statistical Interpretation
01:37Statistical Significance