Applying pragmatic approaches to complex program evaluation: A case study of implementation of the New South Wales...

Complex health promotion programs, which can have multilevels of implementation and multi-components with nonlinear causal pathways, present many evaluation challenges. Traditional evaluation methods often fail to account for the complexity inherent in assessing these programs. In real-world settings, evaluations of complex programs are often beset by additional constraints of limited budgets and short timeframes. Determining whether a complex program is successful and how a program worked requires evaluators of complex programs to adopt a level of pragmatism.

This paper describes a pragmatic evaluation approach used to evaluate the Get Healthy at Work workplace health promotion program, implemented in New South Wales, Australia. Using the program as a case study, we describe some key principles for applying a pragmatic evaluation approach and use these principles to develop an appropriate evaluation strategy.

The evaluation includes multiple research methods to assess program outputs and implementation; and identify emergent program impacts, within constrained resources. The evaluation was guided by epistemological flexibility, methodological comprehensiveness and operational practicality.

Health promotion programs, such as state-wide obesity prevention programs, require appropriate evaluation methods which address their inherent complexity amidst the real-world evaluation constraints, and focuses on the essential evaluation needs.

The main complex program evaluation principles are applicable to other multilevel health promotion programs, challenged by methodological and practical or political constraints.