Observational measures of implementer fidelity for a school-based preventive intervention: development, reliability,...

Area of Science:

• Child Psychology
• Educational Interventions
• Behavioral Science

Background:

• Current implementer fidelity measures often lack precision in assessing adherence and competence.
• The link between fidelity and intervention outcomes can be inconsistent, necessitating improved measurement approaches.

Purpose of the Study:

• To evaluate observational methods for measuring implementer fidelity (adherence and competence) in a school-based preventive intervention.
• To examine the relationship between these fidelity constructs and proximal child outcomes.

Main Methods:

• A randomized trial of the Rochester Resilience Project, a school-based intervention for emotion self-regulation.
• Videotaped observations of 10 paraprofessional implementers delivering lessons to first-third graders.
• Multilevel modeling to assess variance in adherence and competence attributed to implementer and child.

Main Results:

• Implementer significantly accounted for variance in both adherence (41%) and competence (68%), indicating stable implementer qualities.
• Adherence and competence scores were moderately correlated (r=.68).
• Higher adherence and competence predicted improved child and parent-reported intervention response after 6 months, but not teacher-reported externalizing behavior.

Conclusions:

• Observational methods provide a robust way to measure implementer fidelity, capturing both adherence and competence.
• Adherence and competence are vital components of fidelity that influence intervention effectiveness.
• These findings support the use of multiple observations for accurate fidelity assessment in similar programs.