Matching Evaluation Approaches to Levels of Complexity

Area of Science:

• Evaluation science
• Complex interventions research
• Program evaluation methodologies

Background:

• Current evaluation debates persist around outdated positivist-constructivist paradigms.
• Lack of a clear theoretical framework leads to misuse of evaluation approaches by researchers and practitioners.

Purpose of the Study:

• To adapt classic evaluation approaches to contemporary managerial demands and increased complexity.
• To provide an operational tool for categorizing evaluations and matching approaches to objectives and circumstances.
• To address the misuse of evaluation methods due to a lack of conceptual clarity.

Main Methods:

• Utilized three case studies to demonstrate appropriate and inappropriate use of evaluation approaches based on nonlinearity levels.
• Incorporated extensive interviews with program management, staff, and evaluators across continents.
• Supplemented qualitative data with a quantitative survey for robust analysis.

Main Results:

• Evaluation approaches incongruent with nonlinearity levels can impede accurate results assessment.
• Mismatched methods lead to dissatisfaction among evaluation commissioners regarding process and outcomes.
• Separating nonlinear and structural complexity elements enables better categorization and matching of evaluation approaches.

Conclusions:

• Aligning evaluation approaches with program nonlinearity is essential for accurate assessment and stakeholder satisfaction.
• An extended categorization of evaluation approaches, considering nonlinearity, improves the evaluation of complex programs.
• The study offers a practical tool for selecting appropriate evaluation strategies in complex intervention settings.