Multilevel confirmatory factor analysis of the Effective Behavior Support Self-Assessment Survey

One evaluation measure available through the Positive Behavioral Interventions and Supports framework is the Effective Behavior Support Self-Assessment Survey (SAS). Evaluations of the SAS have supported its factor structure. However, the SAS is designed to be completed by school personnel who are nested within other levels of analysis (e.g., schools, grade level, district). There have been no studies examining the SAS from a multilevel perspective. The current study addressed this gap by evaluating the SAS using data from 1,726 respondents across 36 public schools in 3 school districts. Another purpose of this study was to evaluate whether the SAS can be shortened yet remain psychometrically sound to encourage completion of the survey by busy school staff members. Intraclass correlation coefficients indicated substantial dependence in item responses. Factor correlations were substantial across all models tested. Compared with a 2nd-order, hierarchical model and the basic 4-factor model based on the original hypothesized structure of the SAS, the best combination of fit and parsimony was obtained from a reduced set of 24 of the original 46 items organized within a multilevel bifactor structure (e.g., comparative fit index = .979, root-mean-square error of approximation = .019), suggesting that the SAS may be shortened. The bifactor model partitioned item variances to a general implementation fidelity factor and to 1 of the 4 hypothesized factors. These results suggest that the SAS items aim to describe dimensions for measuring implementation status that may be as relevant at the school level as they are at the individual level. (PsycINFO Database Record (c) 2019 APA, all rights reserved).