Related Experiment Video
Updated: Apr 19, 2026

Inverse Probability of Treatment Weighting Propensity Score using the Military Health System Data Repository and National Death Index
Published on: January 8, 2020
Interpreting the results of a retrospective comparison of test and reference treatments in a randomized clinical
1AMF Consulting, Inc., Los Angeles, CA, USA, fmoshe@amf-consulting.com.
Background And Objectives:
The retrospective comparison of test and reference treatment arms in a randomized prospective clinical trial is potentially useful in economic modeling seeking to assess the cost effectiveness of alternative therapies.
Methods:
To enhance the credibility of such retrospective comparisons, we propose the application of the following adjustments to significance levels obtained from standard statistical methodology: (1) a significance test for the lower bound of the 95 % confidence interval for the observed difference, (2) a conservative Bonferroni method of adjustment for multiple comparisons, (3) an adjusted p-value calculated using Scheffe's single-step method, and (4) Bayesian 95 % credibility intervals with a prior centered at zero.
Results:
These adjustments were applied to data from a randomized double-blind concurrent trial (SPD489-325) that established the efficacy and safety of lisdexamfetamine dimesylate (LDX) in children and adolescents with attention-deficit/hyperactivity disorder (ADHD). Prospectively planned analyses demonstrated that the reduction in the symptoms of ADHD was significantly greater than placebo in patients treated with either LDX or the reference treatment, osmotic-release oral system methylphenidate (OROS-MPH). Retrospective analyses showed that the improvement in the symptoms of ADHD was greater in patients treated with LDX than OROS-MPH. We now show that this observation remained significant after the application of the four statistical penalties.
Conclusions:
By adjusting the significance level, it is possible to compare quantitatively such retrospective results with prospectively defined comparisons. However, the qualitative level of such retrospective evidence should remain secondary to that obtained from prospectively specified comparisons in a randomized clinical trial.
Related Concept Videos
Crossover Experiments
Crossover designs are performed even with smaller sample sizes since the samples can act as their controls. These are better than simple randomized trials since patients are exposed to all the treatments.
Bioequivalence Experimental Study Designs: Repeated Measures, Cross-Over, Carry-Over, and Latin Square Designs
Types of Biopharmaceutical Studies: Controlled and Non-Controlled Approaches
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast,...
Blinding
Comparing the Survival Analysis of Two or More Groups
Blind Procedures
