How estimating nuisance parameters can reduce the variance (with consistent variance estimation)

We often estimate a parameter of interest $\psi$ when the identifying conditions involve a finite-dimensional nuisance parameter $\theta \in {ℝ}^d$ . Examples from causal inference are inverse probability weighting, marginal structural models and structural nested models, which all lead to unbiased estimating equations. This article presents a consistent sandwich estimator for the variance of estimators $\hat{\psi }$ that solve unbiased estimating equations including $\theta$ which is also estimated by solving unbiased estimating equations. This article presents four additional results for settings where $\hat{\theta }$ solves (partial) score equations and $\psi$ does not depend on $\theta$ . This includes many causal inference settings where $\theta$ describes the treatment probabilities, missing data settings where $\theta$ describes the missingness probabilities, and measurement error settings where $\theta$ describes the error distribution. These four additional results are: (1) Counter-intuitively, the asymptotic variance of $\hat{\psi }$ is typically smaller when $\theta$ is estimated. (2) If estimating $\theta$ is ignored, the sandwich estimator for the variance of $\hat{\psi }$ is conservative. (3) A consistent sandwich estimator for the variance of $\hat{\psi }$ . (4) If $\hat{\psi }$ with the true $\theta$ plugged in is efficient, the asymptotic variance of $\hat{\psi }$ does not depend on whether $\theta$ is estimated. To illustrate we use observational data to calculate confidence intervals for (1) the effect of cazavi versus colistin on bacterial infections and (2) how the effect of antiretroviral treatment depends on its initiation time in HIV-infected patients.