Related Experiment Video
Updated: Feb 27, 2026
Project-Based Learning Guidelines for Health Sciences Students: An Analysis with Data Mining and Qualitative Techniques
Published on: December 9, 2022
Trends in P Value, Confidence Interval, and Power Analysis Reporting in Health Professions Education Research
Eduardo F Abbott1, Valentina P Serrano, Melissa L Rethlefsen
1E.F. Abbott is a research fellow, Mayo Clinic Multidisciplinary Simulation Center, Mayo Clinic College of Medicine, Rochester, Minnesota, and adjunct instructor of internal medicine, Department of Internal Medicine, Escuela de Medicina, Pontificia Universidad Catolica de Chile, Santiago, Chile. V.P. Serrano is a research fellow, Knowledge and Evaluation Research Unit, Division of Endocrinology, Diabetes, Metabolism and Nutrition, Mayo Clinic, Rochester, Minnesota, and assistant professor, Department of Nutrition, Diabetes and Metabolism, Escuela de Medicina, Pontificia Universidad Catolica de Chile, Santiago, Chile. M.L. Rethlefsen is deputy director and associate librarian, Spencer S. Eccles Health Sciences Library, and section director, Systematic Review Core, Population Health Research Foundation for Discovery, Center for Clinical and Translational Science, University of Utah, Salt Lake City, Utah. T.K. Pandian is a postgraduate year six resident, Department of Surgery, Mayo Clinic College of Medicine, Rochester, Minnesota. N.D. Naik is a postgraduate year four resident, Department of Surgery, Mayo Clinic College of Medicine, Rochester, Minnesota. C.P. West is professor of medicine, professor of biostatistics, and professor of medical education; associate program director, Internal Medicine Residency Program; and consultant, Division of General Internal Medicine, Mayo Clinic College of Medicine, Rochester, Minnesota. V.S. Pankratz is professor of internal medicine, University of New Mexico Health Sciences Center, Albuquerque, New Mexico. D.A. Cook is professor of medicine and professor of medical education; research chair, Mayo Clinic Multidisciplinary Simulation Center; director of research, Office of Applied Scholarship and Education Science; and consultant, Division of General Internal Medicine, Mayo Clinic College of Medicine, Rochester, Minnesota; ORCID: http://orcid.org/0000-0003-2383-4633.
Purpose:
To characterize reporting of P values, confidence intervals (CIs), and statistical power in health professions education research (HPER) through manual and computerized analysis of published research reports.
Method:
The authors searched PubMed, Embase, and CINAHL in May 2016, for comparative research studies. For manual analysis of abstracts and main texts, they randomly sampled 250 HPER reports published in 1985, 1995, 2005, and 2015, and 100 biomedical research reports published in 1985 and 2015. Automated computerized analysis of abstracts included all HPER reports published 1970-2015.
Results:
In the 2015 HPER sample, P values were reported in 69/100 abstracts and 94 main texts. CIs were reported in 6 abstracts and 22 main texts. Most P values (≥77%) were ≤.05. Across all years, 60/164 two-group HPER studies had ≥80% power to detect a between-group difference of 0.5 standard deviations. From 1985 to 2015, the proportion of HPER abstracts reporting a CI did not change significantly (odds ratio [OR] 2.87; 95% CI 1.04, 7.88) whereas that of main texts reporting a CI increased (OR 1.96; 95% CI 1.39, 2.78). Comparison with biomedical studies revealed similar reporting of P values, but more frequent use of CIs in biomedicine. Automated analysis of 56,440 HPER abstracts found 14,867 (26.3%) reporting a P value, 3,024 (5.4%) reporting a CI, and increased reporting of P values and CIs from 1970 to 2015.
Conclusions:
P values are ubiquitous in HPER, CIs are rarely reported, and most studies are underpowered. Most reported P values would be considered statistically significant.
Related Concept Videos
01:09Decision Making: P-value Method
First, a specific claim about the population parameter is proposed. The claim is based on the research question and is stated in a simple form. Further, an opposing statement to the claim is also stated. These statements can act as null and alternative hypotheses: a null hypothesis would be a neutral statement while the alternative hypothesis can...
01:10P-value
P-value stands for the probability value. P-value is the probability that, if the null hypothesis is true, the results from another randomly selected sample will be as extreme or more extreme as the results obtained from the given sample.
A large P-value calculated from the data indicates to not reject the null hypothesis. But a higher P-value does not mean that the null hypothesis is true. The smaller the P-value, the more...
01:23Types of Biopharmaceutical Studies: Controlled and Non-Controlled Approaches
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast,...
01:24Testing a Claim about Population Proportion
There are two methods of testing a claim about a population proportion: (1) Using the sample proportion from the data where a binomial distribution is approximated to the normal distribution and (2) Using the binomial probabilities calculated from the data.
The first method uses normal distribution as an approximation to the binomial distribution. The requirements are as follows: sample size is large...
01:19Interpretation of Confidence Intervals
Confidence intervals have confidence coefficients that are crucial for their interpretation. The most common confidence coefficients are 0.90, 0.95, and 0.99, which can be written as percentages–90%, 95%, and 99%, respectively.
Suppose a person calculates a confidence interval with a confidence coefficient of 0.95. In that case, they can...
01:14Errors In Hypothesis Tests