Using rapid evaluation methods to assess service delivery changes: Lessons learned for evaluation practice during the...

Area of Science:

• Public Health
• Health Services Research
• Evaluation Science

Background:

• The COVID-19 pandemic necessitated rapid changes in government service delivery.
• Evaluators were challenged to provide timely insights into the impact of these adaptations.
• Existing evaluation models were often too slow for the pandemic's pace.

Purpose of the Study:

• To share lessons learned from applying a novel rapid evaluation method (REM) during the COVID-19 pandemic.
• To assess the effectiveness and scalability of practice changes in health and human services.
• To inform evidence-based policy and decision-making in time-constrained settings.

Main Methods:

• Utilized a team-based, mixed-methods design for rapid evaluation.
• Executed evaluations over a 6-8 week period.
• Employed customized rubrics to assess practice changes rigorously.

Main Results:

• Provided real-time insights for 15 program and practice adaptations in Victorian health and human services.
• Demonstrated the practical applicability of balancing rigor with timeliness in evaluations.
• Identified strengths and limitations of a fast-paced evaluation framework.

Conclusions:

• Rapid evaluation methods offer a pragmatic approach for generating timely, evidence-based information.
• Meaningful insights can be achieved through adapted, smaller-scale evaluations.
• Findings can guide evaluative practices in resource and time-limited environments.