Related Experiment Video
Updated: May 26, 2026
06:55
Inverse Probability of Treatment Weighting (Propensity Score) using the Military Health System Data Repository and National Death Index
Published on: January 8, 2020
Comparisons against baseline within randomised groups are often used and can be highly misleading
J Martin Bland1, Douglas G Altman
1University of York, Heslington, York, YO10 5DD, UK. martin.bland@york.ac.uk
Trials
|December 24, 2011
Summary
Researchers often incorrectly compare baseline and final measurements separately within randomized groups. This invalid statistical approach leads to misleading conclusions and inflated Type I error rates, necessitating direct two-sample comparisons.
Area of Science:
- Biostatistics
- Clinical Trial Design
- Statistical Inference
Background:
- Randomized trials frequently employ incorrect statistical methods.
- Some researchers compare baseline to final measurements within each group separately, rather than comparing groups directly.
Purpose of the Study:
- To demonstrate the invalidity of comparing baseline to final measurements separately within randomized groups.
- To highlight the potential for misleading conclusions arising from this flawed methodology.
Main Methods:
- Illustrative examples of the incorrect procedure are provided.
- Statistical simulations were used to assess the validity of the method.
- Algebraic methods were employed to demonstrate the procedure's invalidity.
Main Results:
- The separate comparison of baseline to final measurements is biased and invalid.
- This approach can produce highly misleading study conclusions.
- The actual alpha level can reach 0.50 for two groups and 0.75 for three, indicating a high risk of false positives.
Conclusions:
- Direct two-sample comparison methods are essential for analyzing randomized groups.
- Separate significance tests against baseline are highly misleading and should be avoided.
Related Concept Videos
01:23Types of Biopharmaceutical Studies: Controlled and Non-Controlled Approaches
Biopharmaceutical studies constitute a vital field aiming to enhance drug delivery methods and refine therapeutic approaches, drawing upon diverse interdisciplinary knowledge. In research methodologies, the choice between controlled and non-controlled studies significantly influences the study's reliability and accuracy.
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast, controlled...
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast, controlled...
01:20Comparing the Survival Analysis of Two or More Groups
Survival analysis is a cornerstone of medical research, used to evaluate the time until an event of interest occurs, such as death, disease recurrence, or recovery. Unlike standard statistical methods, survival analysis is particularly adept at handling censored data—instances where the event has not occurred for some participants by the end of the study or remains unobserved. To address these unique challenges, specialized techniques like the Kaplan-Meier estimator, log-rank test, and Cox...
01:52Regression Toward the Mean
Regression toward the mean (“RTM”) is a phenomenon in which extremely high or low values—for example, and individual’s blood pressure at a particular moment—appear closer to a group’s average upon remeasuring. Although this statistical peculiarity is the result of random error and chance, it has been problematic across various medical, scientific, financial and psychological applications. In particular, RTM, if not taken into account, can interfere when researchers try to extrapolate results...
01:13Randomized Experiments
The randomization process involves assigning study participants randomly to experimental or control groups based on their probability of being equally assigned. Randomization is meant to eliminate selection bias and balance known and unknown confounding factors so that the control group is similar to the treatment group as much as possible. A computer program and a random number generator can be used to assign participants to groups in a way that minimizes bias.
Simple randomization
Simple...
Simple randomization
Simple...
01:11Blinding
Blinding is a commonly used method of not telling participants which treatment a subject is receiving. Blinding is a critical part of a randomized control trial or RCT. It reduces the bias that affects the results. In an RCT, blinding is used in the form of a placebo. A placebo effect occurs when untreated subjects falsely believe they have received the treatment and report improved symptoms. A placebo or a dummy treatment is administered to subjects to negate the bias caused by such an effect.
02:07Blind Procedures
Ideally, the people who observe and record the children’s behavior are unaware of who was assigned to the experimental or control group, in order to control for experimenter bias. Experimenter bias refers to the possibility that a researcher’s expectations might skew the results of the study. Remember, conducting an experiment requires a lot of planning, and the people involved in the research project have a vested interest in supporting their hypotheses. If the observers knew which child was...