Related Experiment Video
Updated: Aug 6, 2026
Inverse Probability of Treatment Weighting (Propensity Score) using the Military Health System Data Repository and National Death Index
Published on: January 8, 2020
Treatment effect estimation by comparing observed and predicted outcomes: conditions for valid inference and
Lotta M Meijerink1, Artuur M Leeuwenberg2, Jungyeon Choi2
1Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Universiteitsweg 100, 3508 GA, Utrecht, The Netherlands. l.m.meijerink-20@umcutrecht.nl.
Background:
One non-randomized approach to estimate the average effect of a newly introduced treatment is to compare observed outcomes under the new treatment with predicted outcomes under the standard treatment. These counterfactual predictions are made using a model developed before the new treatment was introduced, using patient characteristics and individualized treatment information. Although the approach has been intuitively applied (e.g., as model-based clinical evaluation in radiotherapy) and can be recognized as a specific case of standardization, a method of virtual controls or a g-method, the theory and conditions required for unbiased treatment effect estimation have not been formally described. The objective of this paper is to formalize the approach and clarify these conditions.
Methods:
We formalize the approach within the potential outcomes framework for causal inference. We explain the methodology, its necessary conditions, and approaches for assessing their validity. These conditions are furthermore illustrated through a case study from radiotherapy, estimating the benefit of proton therapy compared to photon therapy on dysphagia in patients with head and neck cancer.
Results:
We describe a set of five sufficient conditions, including examples of violations: transportability, ignorability of treatment assignment, consistency, positivity, and correct model specification. While these conditions are largely untestable, we describe how empirical evidence, such as comparing predicted and observed outcomes in related samples, can increase confidence in their plausibility.
Conclusion:
When the prediction model predicts well in relevant (sub)populations, the approach can yield unbiased treatment effect estimates. However, there are many possible sources of bias. Therefore, we recommend systematic consideration of all required conditions, informed by domain expertise, and the use of empirical evidence whenever possible to support their plausibility.
Related Concept Videos
01:52Regression Toward the Mean
01:12What is an Experiment?
01:03Prediction Intervals
However, the point estimate is most likely not the exact value of the population parameter, but close to it. After calculating point estimates, we construct interval estimates, called confidence intervals or prediction intervals. This prediction interval comprises a range of values unlike the point estimate and is a better predictor of the observed sample value, y.
The...
01:20Comparing the Survival Analysis of Two or More Groups
01:23Types of Biopharmaceutical Studies: Controlled and Non-Controlled Approaches
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast, controlled...
01:09Odds Ratio