Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Decision Making: P-value Method01:09

Decision Making: P-value Method

5.4K
The process of hypothesis testing based on the P-value method includes calculating the P- value using the sample data and interpreting it.
First, a specific claim about the population parameter is proposed. The claim is based on the research question and is stated in a simple form. Further, an opposing statement to the claim  is also stated. These statements can act as null and alternative hypotheses:  a null hypothesis would be a neutral statement while the alternative hypothesis can...
5.4K
Statistical Inference Techniques in Hypothesis Testing: Parametric Versus Nonparametric Data01:16

Statistical Inference Techniques in Hypothesis Testing: Parametric Versus Nonparametric Data

130
Statistical inference techniques, paramount in hypothesis testing, differentiate into two broad categories: parametric and nonparametric statistics.
Parametric statistics, as the name suggests, assumes that data follow a specific distribution, often a normal distribution. This assumption enables robust hypothesis testing and estimation. Parametric methods, like the Student's t-test or Goodness-of-fit test, are frequently employed in biostatistics due to their robustness. For instance,...
130
Strategies for Assessing and Addressing Confounding01:25

Strategies for Assessing and Addressing Confounding

101
Confounding is a critical issue in epidemiological studies, often leading to misleading conclusions about associations between exposures and outcomes. It occurs when the relationship between the exposure and the outcome is mixed with the effects of other factors that influence the outcome. Given that, addressing confounding is of high importance for drawing accurate inferences in research.
Confounding can be addressed at both the design phase of a study and through analytical methods after data...
101
Statistical Hypothesis Testing01:16

Statistical Hypothesis Testing

1.9K
Hypothesis testing is a critical statistical procedure facilitating informed, evidence-based decisions. It begins with a hypothesis, which is a tentative explanation, or a prediction about a population parameter. This hypothesis can be either a null hypothesis (H0), indicating no effect or difference, or an alternative hypothesis (Ha), suggesting an effect or difference.
Statistical significance measures the probability that an observed result occurred by chance. If this probability, known as...
1.9K
Accuracy and Errors in Hypothesis Testing01:13

Accuracy and Errors in Hypothesis Testing

200
Hypothesis testing is a fundamental statistical tool that begins with the assumption that the null hypothesis H0 is true. During this process, two types of errors can occur: Type I and Type II. A Type I error refers to the incorrect rejection of a true null hypothesis, while a Type II error involves the failure to reject a false null hypothesis.
In hypothesis testing, the probability of making a Type I error, denoted as α, is commonly set at 0.05. This significance level indicates a 5%...
200
Decision Making: Traditional Method01:14

Decision Making: Traditional Method

4.0K
The process of hypothesis testing based on the traditional method includes calculating the critical value, testing the value of the test statistic using the sample data, and interpreting these values.
First, a specific claim about the population parameter is decided based on the research question and is stated in a simple form. Further, an opposing statement to this claim is also stated. These statements can act as null and alternative hypotheses, out of which a null hypothesis would be a...
4.0K

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Effect of a portion-size default nudge on meat consumption and diner satisfaction: controlled experiments in Stanford University dining halls.

BMC public health·2025
Same author

Pitfalls of imputing using incomplete auxiliary variables.

American journal of epidemiology·2025
Same author

Toward evidence-based communication on overweight body mass index and mortality.

BMC medicine·2024
Same author

Assessing robustness to worst case publication bias using a simple subset meta-analysis.

BMJ (Clinical research ed.)·2024
Same author

Sensitivity analysis for the interactive effects of internal bias and publication bias in meta-analyses.

Research synthesis methods·2023
Same author

Effects of an educational planetary plate graphic on meat consumption in a Stanford University dining hall: a randomized controlled trial.

BMC nutrition·2023

Related Experiment Video

Updated: Jul 4, 2025

Meta-analysis of Voxel-Based Neuroimaging Studies using Seed-based d Mapping with Permutation of Subject Images SDM-PSI
06:26

Meta-analysis of Voxel-Based Neuroimaging Studies using Seed-based d Mapping with Permutation of Subject Images SDM-PSI

Published on: November 27, 2019

70.8K

P-hacking in meta-analyses: A formalization and new meta-analytic methods.

Maya B Mathur1

  • 1Quantitative Sciences Unit, Department of Medicine, Stanford University, Stanford, California, USA.

Research Synthesis Methods
|January 25, 2024
PubMed
Summary

Publication bias can occur across studies (SAS) or within studies (SWS) via p-hacking. New methods analyze nonaffirmative estimates to address both selection types, improving meta-analysis robustness.

Keywords:
Bayesian analysisdata dredgingfile drawerselective reportingtruncation

More Related Videos

Author Spotlight: Evaluating the Adjuvant Efficacy and Safety of Angong Niuhuang Pill in Viral Encephalitis Treatment
08:36

Author Spotlight: Evaluating the Adjuvant Efficacy and Safety of Angong Niuhuang Pill in Viral Encephalitis Treatment

Published on: April 19, 2024

526
Drug Repurposing Hypothesis Generation Using the "RE:fine Drugs" System
05:10

Drug Repurposing Hypothesis Generation Using the "RE:fine Drugs" System

Published on: December 11, 2016

9.6K

Related Experiment Videos

Last Updated: Jul 4, 2025

Meta-analysis of Voxel-Based Neuroimaging Studies using Seed-based d Mapping with Permutation of Subject Images SDM-PSI
06:26

Meta-analysis of Voxel-Based Neuroimaging Studies using Seed-based d Mapping with Permutation of Subject Images SDM-PSI

Published on: November 27, 2019

70.8K
Author Spotlight: Evaluating the Adjuvant Efficacy and Safety of Angong Niuhuang Pill in Viral Encephalitis Treatment
08:36

Author Spotlight: Evaluating the Adjuvant Efficacy and Safety of Angong Niuhuang Pill in Viral Encephalitis Treatment

Published on: April 19, 2024

526
Drug Repurposing Hypothesis Generation Using the "RE:fine Drugs" System
05:10

Drug Repurposing Hypothesis Generation Using the "RE:fine Drugs" System

Published on: December 11, 2016

9.6K

Area of Science:

  • Biostatistics
  • Research Methodology
  • Scientific Integrity

Background:

  • Publication bias, traditionally viewed as selection across studies (SAS), favors publishing significant results.
  • Selection within studies (SWS), or p-hacking, can bias estimates even among published affirmative findings.
  • Existing meta-analysis methods often fail to account for the combined impact of SAS and SWS.

Purpose of the Study:

  • To develop novel methods for meta-analysis that address both selection across studies (SAS) and selection within studies (SWS).
  • To provide tools for assessing the robustness of meta-analyses against combined publication bias.
  • To improve the accuracy and reliability of meta-analytic estimates in the presence of complex selection biases.

Main Methods:

  • Proposed two new analysis methods focusing exclusively on published nonaffirmative (negative or nonsignificant) estimates.
  • Developed "right-truncated meta-analysis" (RTMA) to estimate the meta-analytic mean by imputing the full distribution of effects.
  • Introduced "meta-analysis of nonaffirmative studies" (MAN) as a conservative estimate under weakened assumptions.

Main Results:

  • The proposed methods, RTMA and MAN, offer complementary approaches to handle joint SAS and SWS.
  • RTMA provides an estimate of the underlying meta-analytic mean by modeling the entire distribution of population effects.
  • MAN yields a conservative, negatively biased estimate, offering robustness under broader conditions.

Conclusions:

  • The developed methods and accompanying R package (phacking) enhance the ability to detect and correct for complex publication bias.
  • These approaches supplement existing techniques, providing a more comprehensive assessment of meta-analysis validity.
  • Addressing both SAS and SWS is crucial for robust scientific conclusions derived from aggregated research.