Accountability in Intervention Research: Developing a Fidelity Checklist of a Mental Health Intervention in Prisons

Adherence to intervention content and delivery protocols is vital in establishing the efficacy of treatment programs for mental illnesses. Using a fidelity tool during interventions can substantially increase the likelihood of clients receiving the most scientifically rigorous treatment. This article outlines the steps taken to develop a fidelity checklist to measure treatment adherence of a two-part intervention delivered in a prison setting. Researchers followed the five-step guide by Feely et al. (Child and Adolescent Social Work Journal, 35(2), 139-152: 2018) and describe the process of developing a fidelity tool to measure treatment adherence to a newly adapted CBT-based intervention designed to maximize uptake for participants with serious mental illnesses. Key decision points are discussed, along with final decisions and contextual considerations. A 26-item checklist was developed to measure treatment adherence related to process, content, and adaptations of the intervention. The checklist follows the structure of the CBT intervention, as well as provides flexibility for the delivery adaptations. Pilot testing of the checklist revealed all sessions were implemented with at least 85% fidelity, and 90% of sessions were implemented with at least 90% fidelity. Raters agreed on the fidelity of a session in 99.6% of sessions. Contextual considerations included the highly secure study setting, reconciling the constant monitoring of a group and creating a treatment environment, the flexibility mandated by the intervention, the relative newness of the intervention, and the limitations based on study aims and resources. These results illustrate how study specific considerations and challenges can be successfully navigated in the development and deployment of a fidelity tool in a real-world setting. The fidelity checklist achieved our goal of measuring treatment adherence for this intervention. In the development of a fidelity tool, we recommend leaving space for raters to note specific considerations that disrupt facilitators' ability to deploy the intervention precisely. Measuring fidelity is imperative for mental health interventions to ensure that the treatment is responsible for the changes observed in clients.