Jove
Visualize
Contact Us

Related Concept Videos

Randomized Experiments01:13

Randomized Experiments

9.3K
The randomization process involves assigning study participants randomly to experimental or control groups based on their probability of being equally assigned. Randomization is meant to eliminate selection bias and balance known and unknown confounding factors so that the control group is similar to the treatment group as much as possible. A computer program and a random number generator can be used to assign participants to groups in a way that minimizes bias.
Simple randomization
Simple...
9.3K
Decision Making: P-value Method01:09

Decision Making: P-value Method

7.4K
The process of hypothesis testing based on the P-value method includes calculating the P- value using the sample data and interpreting it.
First, a specific claim about the population parameter is proposed. The claim is based on the research question and is stated in a simple form. Further, an opposing statement to the claim  is also stated. These statements can act as null and alternative hypotheses:  a null hypothesis would be a neutral statement while the alternative hypothesis can...
7.4K
Bonferroni Test01:10

Bonferroni Test

3.6K
The Bonferroni test is a statistical test named after Carlo Emilio Bonferroni, an Italian mathematician best known for Bonferroni inequalities. This statistical test is a type of multiple comparison test to determine which means are different than the rest. Bonferroni test can minimize the Type 1 error by reducing the significance level alpha, which otherwise increases with sample pairs.
The means of different samples are first paired in all possible combinations.
The null hypothesis of the...
3.6K
P-value01:10

P-value

9.6K
P-value is one of the most crucial concepts in statistics.
P-value stands for the probability value.  P-value is the probability that, if the null hypothesis is true, the results from another randomly selected sample will be as extreme or more extreme as the results obtained from the given sample.
A large P-value calculated from the data indicates to  not reject the null hypothesis. But a higher P-value does not mean that the null hypothesis is true. The smaller the P-value, the more...
9.6K
Testing a Claim about Population Proportion01:24

Testing a Claim about Population Proportion

4.2K
A complete procedure for testing a claim about a population proportion is provided here.
There are two methods of testing a claim about a population proportion: (1) Using the sample proportion from the data where a binomial distribution is approximated to the normal distribution and (2) Using the binomial probabilities calculated from the data.
The first method uses normal distribution as an approximation to the binomial distribution. The requirements are as follows: sample size is large...
4.2K
Friedman Two-way Analysis of Variance by Ranks01:21

Friedman Two-way Analysis of Variance by Ranks

599
Friedman's Two-Way Analysis of Variance by Ranks is a nonparametric test designed to identify differences across multiple test attempts when traditional assumptions of normality and equal variances do not apply. Unlike conventional ANOVA, which requires normally distributed data with equal variances, Friedman's test is ideal for ordinal or non-normally distributed data, making it particularly useful for analyzing dependent samples, such as matched subjects over time or repeated measures...
599

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Contrast-specific propensity scores for causal inference with multiple interventions.

Statistical methods in medical research·2024
Same author

Estimating adjusted risk differences by multiply-imputing missing control binary potential outcomes following propensity score-matching.

Statistics in medicine·2021
Same author

Automatic detection of influential actors in disinformation networks.

Proceedings of the National Academy of Sciences of the United States of America·2021
Same author

Nonstandard conditionally specified models for nonignorable missing data.

Proceedings of the National Academy of Sciences of the United States of America·2020
Same author

Catalytic prior distributions with application to generalized linear models.

Proceedings of the National Academy of Sciences of the United States of America·2020
Same author

Covariate-adjusted survival analyses in propensity-score matched samples: Imputing potential time-to-event outcomes.

Statistical methods in medical research·2018
Same journal

Predictor-Assisted Nonparametric Graphical Models With Multivariate Error-Prone Data.

Statistics in medicine·2026
Same journal

Optimizing Treatment Decision Estimation for Right-Censored Survival Data Through Parameter Transfer Learning.

Statistics in medicine·2026
Same journal

Latent Class Log-Linear Models for Estimating Diagnostic Test Accuracy Without a Gold Standard: A Simulation Study.

Statistics in medicine·2026
Same journal

Interpretable Bayesian Modeling for Multireader Multicase Studies: Addressing Overdispersion and Limited Sample Size in Diagnostic Enhancement Evaluation.

Statistics in medicine·2026
Same journal

Adaptive Sequential Multiple Hypotheses Testing for Concomitant Vaccine Safety Surveillance.

Statistics in medicine·2026
Same journal

Novel Distance Regression for Repeated Outcomes With Missing Data: Applications to Longitudinal and Crossover Studies of Microbiome Beta-Diversity.

Statistics in medicine·2026
See all related articles
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Experiment Video

Updated: Apr 12, 2026

Generalized Psychophysiological Interaction PPI Analysis of Memory Related Connectivity in Individuals at Genetic Risk for Alzheimer's Disease
09:38

Generalized Psychophysiological Interaction PPI Analysis of Memory Related Connectivity in Individuals at Genetic Risk for Alzheimer's Disease

Published on: November 14, 2017

15.9K

Valid randomization-based p-values for partially post hoc subgroup analyses.

Joseph J Lee1, Donald B Rubin1

  • 1Department of Statistics, Harvard University, 1 Oxford St., Cambridge, MA, 02138, U.S.A.

Statistics in Medicine
|May 12, 2015
PubMed
Summary
This summary is machine-generated.

Partially post-hoc subgroup analyses compare existing data to new subgroup data, often leading to invalid statistical conclusions. A new randomization-based method offers valid posterior predictive p-values for these subgroup analyses.

Keywords:
Fisher randomization testcausal inferencemultiple comparisonsposterior predictive p-valuestatistical significance

More Related Videos

Inverse Probability of Treatment Weighting Propensity Score using the Military Health System Data Repository and National Death Index
06:55

Inverse Probability of Treatment Weighting Propensity Score using the Military Health System Data Repository and National Death Index

Published on: January 8, 2020

15.5K
Candidate Gene Testing in Clinical Cohort Studies with Multiplexed Genotyping and Mass Spectrometry
05:53

Candidate Gene Testing in Clinical Cohort Studies with Multiplexed Genotyping and Mass Spectrometry

Published on: June 21, 2018

10.8K

Related Experiment Videos

Last Updated: Apr 12, 2026

Generalized Psychophysiological Interaction PPI Analysis of Memory Related Connectivity in Individuals at Genetic Risk for Alzheimer's Disease
09:38

Generalized Psychophysiological Interaction PPI Analysis of Memory Related Connectivity in Individuals at Genetic Risk for Alzheimer's Disease

Published on: November 14, 2017

15.9K
Inverse Probability of Treatment Weighting Propensity Score using the Military Health System Data Repository and National Death Index
06:55

Inverse Probability of Treatment Weighting Propensity Score using the Military Health System Data Repository and National Death Index

Published on: January 8, 2020

15.5K
Candidate Gene Testing in Clinical Cohort Studies with Multiplexed Genotyping and Mass Spectrometry
05:53

Candidate Gene Testing in Clinical Cohort Studies with Multiplexed Genotyping and Mass Spectrometry

Published on: June 21, 2018

10.8K

Area of Science:

  • Biostatistics
  • Clinical Trial Design
  • Statistical Inference

Background:

  • Partially post-hoc subgroup analyses, which compare existing randomized data to new subgroup-specific experimental data, are increasingly used.
  • These analyses can lead to statistical debates regarding treatment efficacy and device performance, as seen in a medical device example.
  • The validity of conclusions drawn from such analyses is often questionable due to their inherent biases.

Purpose of the Study:

  • To clarify the statistical invalidity stemming from partially post-hoc subgroup analyses.
  • To propose a novel randomization-based approach for generating valid posterior predictive p-values for these specific subgroup analyses.
  • To evaluate the operating characteristics of the proposed method through simulations.

Main Methods:

  • Definition and illustration of 'partially post-hoc' subgroup analyses using a motivating example.
  • Identification of the sources of invalidity in these types of analyses.
  • Development of a randomization-based framework for valid posterior predictive p-value generation.
  • Simulation studies to assess the proposed method's performance under null and alternative hypotheses.

Main Results:

  • Partially post-hoc subgroup analyses, as defined, can instigate statistical debate and yield unreliable efficacy conclusions.
  • The proposed randomization-based approach provides a method for generating valid posterior predictive p-values for partially post-hoc subgroups.
  • Simulation results indicate that the new approach possesses desirable statistical properties under various hypothesis scenarios.

Conclusions:

  • The statistical validity of partially post-hoc subgroup analyses is compromised.
  • A novel randomization-based method offers a statistically sound approach for analyzing such subgroups.
  • The proposed method demonstrates potential for improving the reliability of subgroup analyses in clinical research.