A mandala of faculty development: using theory-based evaluation to explore contexts, mechanisms and outcomes

Demonstrating the impact of faculty development, is an increasingly mandated and ever elusive goal. Questions have been raised about the adequacy of current approaches. Here, we integrate realist and theory-driven evaluation approaches, to evaluate an intensive longitudinal program. Our aim is to elucidate how faculty development can work to support a range of outcomes among individuals and sub-systems in the academic health sciences. We conducted retrospective framework analysis of qualitative focus group data gathered from 79 program participants (5 cohorts) over a 10-year period. Additionally, we conducted follow-up interviews with 15 alumni. We represent the interactive relationships among contexts, mechanisms, and outcomes as a "mandala" of faculty development. The mandala illustrates the relationship between the immediate program context, and the broader institutional context of academic health sciences, and identifies relevant change mechanisms. Four primary mechanisms were collaborative-reflection, self-reflection and self-regulation, relationship building, and pedagogical knowledge acquisition. Individual outcomes, including changed teaching practices, are described. Perhaps most interestingly, secondary mechanisms-psychological and structural empowerment-contributed to institutional outcomes through participants' engagement in change leadership in their local contexts. Our theoretically informed evaluation approach models how faculty development, situated in appropriate institutional contexts, can trigger mechanisms that yield a range of benefits for faculty and their institutions. The adopted methods hold potential as a way to demonstrate the often difficult-to-measure outcomes of educational programs, and allow for critical examination as to how and whether faculty development programs can accomplish their espoused goals.