Designing a program evaluation for a medical-dental service for adults with autism and intellectual disabilities...

Robust evaluation of service models can improve the quality and efficiency of care while articulating the models for potential replication. Even though it is an essential part of learning health systems, evaluations that benchmark and sustain models serving adults with developmental disabilities are lacking, impeding pilot programs from becoming official care pathways. Here, we describe the development of a program evaluation for a specialized medical-dental community clinic serving adults with autism and intellectual disabilities in Montreal, Canada.

Using a Participatory Action-oriented approach, researchers and staff co-designed an evaluation for a primary care service for this population. We performed an evaluability assessment to identify the processes and outcomes that were feasible to capture and elicited perspectives at both clinical and health system levels. The RE-AIM framework was used to categorize and select tools to capture data elements that would inform practice at the clinic.

We detail the process of conceptualizing the evaluation framework and operationalizing the domains using a mixed-methods approach. Our experience demonstrated (1) the utility of a comprehensive framework that captures contextual factors in addition to clinical outcomes, (2) the need for validated measures that are not cumbersome for everyday practice, (3) the importance of understanding the functional needs of the organization and building a sustainable data infrastructure that addresses those needs, and (4) the need to commit to an evolving, "living" evaluation in a dynamic health system.

Evaluation employing rigorous patient-centered and systems-relevant metrics can help organizations effectively implement and continuously improve service models. Using an established framework and a collaborative approach provides an important blueprint for a program evaluation in a learning health system. This work provides insight into the process of integrating care for vulnerable populations with chronic conditions in health care systems and integrated knowledge generation processes between research and health systems.