Analyzing and Comparing Univariate, Multivariate, and Bifactor Generalizability Theory Designs for Hierarchically...

We demonstrate how to use structural equation models to represent generalizability theory-based univariate, multivariate, and bifactor model designs. Analyses encompassed multi-occasion data obtained from the recently expanded form of the Big Five Inventory (BFI-2) that measures the broad personality domain constructs Agreeableness, Conscientiousness, Extraversion, Negative Emotionality, and Open-Mindedness along with three nested subdomain facets within each global domain. Results overall highlighted the importance of taking both item and occasion effects into account but underscored additional benefits of the multivariate and bifactor designs in providing more appropriate indices of generalizability for composite scores and effective ways to gauge subscale added value. Bifactor models further extended partitioning of universe score variance to separate general and group factor effects at both composite and subscale levels, expanded score consistency indices to distinguish or combine such effects, and allowed for further evaluation of score dimensionality and subscale viability. We provide guidelines, formulas, and code in R for analyzing all illustrated designs within the article and extended online Supplemental Material.