Communicating Complex STEM Program Evaluation to Diverse Stakeholders

Visual representations, such as pathway models, are increasingly being used to both communicate higher education science, technology, engineering, and mathematics (STEM) education program evaluation plans as well as accurately represent complex programs and the systems within which the educational programs reside. However, these representations can be overwhelming to audiences that are not familiar with the program's structure or engaged in the evaluation process. The goal of this methods essay is to help both evaluators and discipline-based education researchers improve communication about program evaluation with a variety of stakeholders. We propose a three-stage method for developing progressively less complex visualizations to build affordances that help make the program evaluation process and statements of program impact more meaningful to a wider range of audiences. The creation of less complex visualizations can facilitate understanding by allowing a stakeholder to more easily "see" the structure of the program and thereby may evoke a greater willingness to take action and make meaningful programmatic changes based on strategic evaluation planning. To aid readers, we describe how we modified the Systems Evaluation Protocol (SEP) to develop simplified visualizations when evaluating a long-standing college science faculty development program, the Summer Institutes on Scientific Teaching.