Examining heterogeneity in residual variance to detect differential response to treatments

Individual differences in response to treatments have been a long-standing interest in education, psychology, and related fields. This article presents a conceptual framework and hierarchical modeling strategies that may help identify the subgroups for whom, or the conditions under which, a particular treatment is associated with better outcomes. The framework discussed in this article shows how differences in residual dispersion between treatment and control group members can signal omitted individual characteristics that may interact with treatments (Bryk & Raudenbush, 1988) and sensitizes us to individual- and cluster-level confounders of inferences concerning dispersion in quasi-experimental studies in multilevel settings. Based on the framework, hierarchical modeling strategies are developed to uncover interactions between treatments and individual characteristics, which are readily applicable to various settings in multisite evaluation studies. These strategies entail jointly modeling the mean and dispersion structures in hierarchical models. We illustrate the implementation of this framework through fully Bayesian analyses of the data from a study of the effectiveness of a reform-minded mathematics curriculum.