Significance Tests and Estimates for R² for Multiple Regression in Multiply Imputed Datasets: A Cautionary Note on...

Whenever multiple regression is applied to a multiply imputed data set, several methods for combining significance tests for $$ and the change in $$ across imputed data sets may be used: the combination rules by Rubin, the Fisher z-test for $$ by Harel, and F-tests for the change in $$ by Chaurasia and Harel. For pooling $$ itself, Harel proposed a method based on a Fisher z transformation. In the current article, it is argued that the pooled $$ based on the Fisher z transformation, the Fisher z-test for $$ , and the F-test for the change in $$ have some theoretical flaws. An argument is made for using Rubin's method for pooling significance tests for $$ instead, and alternative procedures for pooling $$ are proposed: simple averaging and a pooled $$ constructed from the pooled significance test by Rubin. Simulations show that the Fisher z-test and Chaurasia and Harel's F-tests generally give inflated type-I error rates, whereas the type-I error rates of Rubin's method are correct. Of the methods for pooling the point estimates of $$ no method clearly performs best, but it is argued that the average of $$ 's across imputed data set is preferred.