Design and implementation of an integrated, continuous evaluation, and quality improvement system for a state-based...

Area of Science:

• Public Health
• Program Evaluation
• Health Services Research

Background:

• Statewide home visiting programs aim to improve parenting skills and connect at-risk families to services.
• Continuous Quality Improvement (CQI) and rigorous evaluation are crucial for optimizing program effectiveness.
• Integrating evaluation into daily practice presents implementation challenges.

Purpose of the Study:

• To describe the design and implementation of an evaluation system for a statewide home visiting program.
• To assess the system's progress in meeting intended outputs and short-term outcomes.
• To facilitate both CQI and scientifically rigorous evaluations within the program.

Main Methods:

• A formative evaluation approach was used, examining reach, dose, and fidelity.
• Data collection involved electronic/paper surveys, administrative data, management meeting notes, and medical chart reviews.
• The system design included selecting four process and forty outcome measures tracked via standardized tools.

Main Results:

• Training reach and dose were adequate for system launch, with all 165 staff utilizing it for family management.
• Supervisors (n=18) use the system for routine activity tracking; data quality is sufficient for reviews.
• Within 7 months, the system tracked 3,794 families (7,937 individuals), showing high use and acceptance.

Conclusions:

• Implementing a structured evaluation system with CQI is feasible in large, existing statewide programs.
• The system effectively drives modest improvements in program management.
• High staff acceptance and data sufficiency support its role in ongoing program enhancement.