Evaluating long-term effects of a psychiatric treatment using instrumental variable and matching approaches

Evaluating treatment effects in non-randomized studies is challenging due to the potential unmeasured confounding and complex form of observed confounding. Propensity score based approaches, such as matching or weighting, are commonly used to handle observed confounding variables. The instrumental variable (IV) method is known to guard against unmeasured confounding if a good instrument can be identified. We propose to combine both methods to estimate the long-term treatment effect in a longitudinal psychiatric study. The NIMH collaborative Multi-site Treatment study of children with Attention-deficit/hyperactivity disorder (ADHD) compared different treatment strategies for children diagnosed with ADHD (known as MTA study). The first 14 months is a randomized study and the participants are allowed to choose their desired treatment strategies afterwards. Follow-up measurements are taken at 24, 36 and 72 months. Randomization is often considered as a good instrument since it is not associated with any covariate, observed or unobserved. We first apply a randomization based IV method to estimate the self-selected medication effect on outcome at the end of 72 months. However this approach yields results with huge standard errors due to randomization's weak relationship with later treatment selection. We then consider the self-selection right after the randomization as an instrument, because it is associated with later treatment selection and it is unlikely to affect the outcome directly given the five-year time lapse. To better control the confounding due to observed factors, propensity score matching is used to create a subpopulation with comparable covariate distributions across different self-selected treatments. Using MTA data, matching-enhanced IV estimation yields the most sensible result, while other estimation strategies tend to imply a spurious significant effect. Also, our simulation study shows that the matching-enhanced IV estimation outperforms non-matched methods in terms of relative bias.