Linking classical test theory and two-level hierarchical linear models

This paper considers the link between classical test theory (CTT) and two-level hierarchical linear models (HLM). Conceptualizing that items are nested within subjects, we can reformulate the ANOVA classical test model as an HLM. In this HLM framework, item difficulty parameters are represented by the fixed effects, and subject's abilities are represented by the random effects. The population reliability of either the total or the mean score can be represented by a function of the random effects parameters and the number of items. For estimation, taking advantage of the balanced design nature of CTT, we can obtain explicit formulas for parameter estimates of both fixed and random effects in HLM. It reveals that the formula and the estimate derived from HLM exactly match those of CTT reliability, which are equivalent to Cronbach's coefficient alpha under the assumptions of essentially tau equivalent measures. Not only that, we can obtain most of the important quantities in CTT such as estimates of item difficulty, standard error of measurement, true score, and person ability in a single HLM model. Thus, the CTT model formulated by HLM framework provides a systematic approach on measurement analysis by CTT. For illustrative purpose, a small data set was analyzed using HLM software (Raudenbush, Bryk, Cheong, & Congdon, 2000). The results confirmed the theoretical link between CTT and HLM.