Multilevel confirmatory ordinal factor analysis of the Life Skills Profile-16

The aim of this study was to assess the factor structure of the Life skills profile-16 (LSP-16; Buckingham, Burgess, Solomon, Pirkis, & Eagar, 1998a, 1998b) with a view to meeting the assumption of statistical independence that is at significant risk of violation due to the dependency introduced to the data by pooling numerous ratings made by the same observers across independent patients. The sample consisted of 20,181 outpatients rated by 2,071 clinicians employed within 54 mental health organizations within the New South Wales public adult mental health service. To estimate the extent to which the item scores were contaminated with rater-level intraclass correlations (ICC), I fit 16 3-level multinominal ordered proportional odds intercept only models that revealed large ICCs associated with Level 2 (the rater of the LSP-16) demonstrating that a multilevel analysis was required. A multilevel confirmatory factor analysis (M-CFA) using robust weighted least squares (B. O. Muthén, du Toit, & Spisic, 1997) with polychoric correlation was used to test the fit of 2 measurement models that were hypothesized a priori. The 2 models failed to provide an acceptable fit to the sample data and within- and between-level CFAs were used to inform revisions to the 4-factor model. A 15-item version of the LSP was developed, which provided an improved approximate fit in an M-CFA. Limitations of these findings are discussed.