The need to adapt standardized outcomes measures for community mental health

In order to determine whether evidence-based practices are effective in community settings, the use of measurement tools that accurately capture symptoms and functioning in the community context is imperative. Without accurate measurement, researchers face the troubling possibility that tests of effective treatments may not attain positive results because of measurement error. The authors have experienced apparently serious problems with measurement validity in community mental health practice settings in which we are conducting diffusion research. Features of survey design that strengthen measurement in the university setting (where survey instruments are designed and treatments are usually tested) appear to work against us in community settings. Problems are compounded when it is necessary to measure across several domains, and thus, to employ multiple questionnaires. Here, we discuss the problems we have experienced with three common survey design strategies: (1) the use of multiple, similar items; (2) the use of reverse-coded items; and (3) the use of graded response-items. These strategies result in measures whose length and cognitive complexity may compromise their validity in community settings. Race is an especially important issue in diffusion research, as we lack even basic information concerning efficacy and effectiveness for some important evidence-based treatments. Invalid measures could seriously delay transfer and development of appropriate treatments. Careful psychometric work is badly needed, but such work is time intensive and costly. In the meantime, we encourage researchers to consider the modifications suggested here.