A most stubborn bias: no adjustment method fully resolves confounding by indication in observational studies

To evaluate the effectiveness of methods that control for confounding by indication, we compared breast cancer recurrence rates among women receiving adjuvant chemotherapy with those who did not.

In a medical record review-based study of breast cancer treatment in older women (n=1798) diagnosed between 1990 and 1994, our crude analysis suggested that adjuvant chemotherapy was positively associated with recurrence (hazard ratio [HR]=2.6; 95% confidence interval [CI]=1.9, 3.5). We expected a protective effect, so postulated that the crude association was confounded by indications for chemotherapy. We attempted to adjust for this confounding by restriction, multivariable regression, propensity scores (PSs), and instrumental variable (IV) methods.

After restricting to women at high risk for recurrence (n=946), chemotherapy was not associated with recurrence (HR=1.1; 95% CI=0.7, 1.6) using multivariable regression. PS adjustment yielded similar results (HR=1.3; 95% CI=0.8, 2.0). The IV-like method yielded a protective estimate (HR=0.9; 95% CI=0.2, 4.3); however, imbalances of measured factors across levels of the IV suggested residual confounding.

Conventional methods do not control for unmeasured factors, which often remain important when addressing confounding by indication. PS and IV analysis methods can be useful under specific situations, but neither method adequately controlled confounding by indication in this study.