Expert panel reviews of research centers: the site visit process

Site visits are used extensively in a variety of settings within the evaluation community. They are especially common in making summative value decisions about the quality and worth of research programs/centers. However, there has been little empirical research and guidance about how to appropriately conduct evaluative site visits of research centers. We review the processes of two site visit examples using an expert panel review: (1) a process to evaluate four university research centers and (2) a process to review a federally sponsored research center. A set of 14 categories describing the expert panel review process was obtained through content analysis and participant observation. Most categories were addressed differently through the two processes highlighting the need for more research about the most effective processes to use within different contexts. Decisions about how to structure site visits appear to depend on the research context, practical considerations, the level at which the review is being conducted and the intended impact of the report. Future research pertaining to the selection of site visitors, the autonomy of the visitors in data collection and report writing, and the amount and type of information provided would be particularly valuable.