The Impacts of Ignoring a Crossed Factor in Analyzing Cross-Classified Data

Cross-classified random-effects models (CCREMs) are used for modeling nonhierarchical multilevel data. Misspecifying CCREMs as hierarchical linear models (i.e., treating the cross-classified data as strictly hierarchical by ignoring one of the crossed factors) causes biases in the variance component estimates, which in turn, results in biased estimation in the standard errors of the regression coefficients. Analytical studies were conducted to provide closed-form expressions for the biases. With balanced design data structure, ignoring a crossed factor causes overestimation of the variance components of adjacent levels and underestimation of the variance component of the remaining crossed factor. Moreover, ignoring a crossed factor at the kth level causes underestimation of the standard error of the regression coefficient of the predictor associated with the ignored factor and overestimation of the standard error of the regression coefficient of the predictor at the (k-1)th level. Simulation studies were also conducted to examine the effect of different structures of cross-classification on the biases. In general, the direction and magnitude of the biases depend on the level of the ignored crossed factor, the level with which the predictor is associated at, the magnitude of the variance component of the ignored crossed factor, the variance components of the predictors, the sample sizes, and the structure of cross-classification. The results were further illustrated using the Early Childhood Longitudinal Study-Kindergarten Cohort data.