Propensity score weighting: an application to an Early Head Start dental study

Nonrandomized group assignment in intervention studies can lead to imbalances in preintervention covariates and biased effect estimates. We use propensity score estimation to account for such imbalances in an Early Head Start (EHS) dataset with rich pretreatment information. We compare propensity score results using standard logistic regression models (LRMs) versus generalized boosted models (GBMs).

We estimated propensity scores using 47 socio-demographic characteristics and EHS enrollment criteria obtained by parent interviews from a state-wide sample of 637 EHS and 930 Medicaid-matched control children. LRMs and GBMs were used to estimate propensity scores related to EHS enrollment. Performance of both approaches was evaluated via a) measures of balance of pretreatment covariate distributions between treated and control subjects, and b) stability of propensity score weights measured by the effective sample size.

Distributions of all variables were balanced for EHS and non-EHS groups using propensity score weights calculated with LRM and GBM. Compared with LRM, GBM resulted in better balance between treated and propensity score-weighted control distributions. The effective sample size of the controls decreased from 930 subjects to 507 with GBM and to 335 with LRM.

Although propensity scores derived from GBM and LRM both effectively balanced observed preintervention covariates, GBM resulted in better covariate balance compared with LRM. GBM also resulted in a larger effective sample size of the control group compared with LRM. Propensity score weighting using GBM is an effective statistical method to reduce confounding due to imbalanced distributions of measured preintervention covariates in this EHS intervention study.