Interaction of Theory and Practice to Assess External Validity.

Area of Science:

• Evaluation science
• Health services research
• Intervention science

Background:

• External validity, the ability to generalize treatment effects, is challenged by variations in local contexts.
• Assessing external validity is crucial for decision-makers evaluating and adopting evidence-based interventions.
• Ensuring the quality of interventions as they spread across diverse settings is a key challenge.

Purpose of the Study:

• To outline the rationale for employing program theory in assessing external validity.
• To highlight the value of a more systematic integration of theory and practice in evaluation.
• To demonstrate how theory and practice can enhance the generalizability of intervention findings.

Main Methods:

• Reviewing advancements in external validity, program theory, practitioner expertise, and local adaptation.
• Providing examples of program theory application, adaptation to diverse contexts, and generalization to unstudied settings.
• Describing ongoing work to systematically study treatment variation and context.

Main Results:

• Practitioner experiences, though not systematically sampled, offer valuable insights into diverse contexts, challenging researchers and developers.
• Researchers have developed methods to adapt interventions based on practitioner feedback and contextual understanding.
• Organized practitioner insights can significantly inform the assessment of external validity.

Conclusions:

• Systematic programs of inquiry are more effective at addressing contextual challenges in external validity.
• A systematic interaction between theory and practice can enhance evaluation processes.
• Integrating practitioner expertise with program theory can refine interventions and prioritize future research on context and treatment variation.