Who is the marginal patient? Understanding instrumental variables estimates of treatment effects

To clarify the issues of generalizability arising from the use of instrumental variable (IV) methods to estimate treatment effects in nonexperimental medical outcome studies.

We generate Monte Carlo data designed to resemble typical data sets where detailed health status information is unavailable and the treatment assignment process is unobserved. The model used to generate our data makes the realistic assumption that unobservable health status characteristics of patients influence the treatment assignment process and the effectiveness of treatment.

We use Monte Carlo data to illustrate the circumstances where IV estimates generalize to an unobservable patient subpopulation and those where IV estimates generalize to the entire patient population represented by the sample used in the analysis. We also simulate the effect of two policy changes that affect practice patterns. Further, we show that IV estimates are useful for predicting the effect of these changes on treatment effectiveness when the subpopulation to which the IV estimate refers is the same or very similar to the population whose treatment status is affected by the policy change.

Health services researchers cannot take for granted that IV estimates generalize to the same population represented by the sample used for analysis. Instead, researchers must rely on their knowledge of clinical practice and theory regarding the treatment assignment process in interpreting their results and in predicting the effect of changes in practice patterns.