A Randomized Trial to Identify Accurate Measurement Methods for Adherence to Cognitive-Behavioral Therapy

Area of Science:

• Psychology
• Clinical Psychology
• Implementation Science

Background:

• Clinician fidelity to cognitive behavioral therapy (CBT) is crucial for achieving desired clinical outcomes and ensuring quality of care.
• Existing methods for measuring CBT fidelity are often inefficient, lacking demonstrated reliability and validity.
• Accurate assessment of adherence, a key component of fidelity, is essential for effective CBT implementation.

Purpose of the Study:

• To compare the efficiency and accuracy of three methods for assessing CBT adherence: self-report, chart-stimulated recall, and behavioral rehearsal.
• To determine if these methods yield comparable results to direct observation, the established gold standard for fidelity measurement.
• To identify a potentially more resource-efficient method for assessing CBT fidelity in clinical practice and research.

Main Methods:

• A randomized trial design was employed, with 126 clinicians from community mental health agencies randomized into three groups: self-report, chart-stimulated recall, or behavioral rehearsal.
• Audio-recorded therapy sessions (n=288) from clinicians' caseloads were assessed for fidelity using the assigned method and direct observation.
• Fidelity measures included adherence maximum score, mean observed techniques, and total count of observed techniques, analyzed using three-level mixed effects regression models.

Main Results:

• Behavioral rehearsal produced adherence scores comparable to direct observation, with no statistically significant differences.
• Self-report and chart-stimulated recall methods significantly overestimated adherence compared to direct observation (p < .01).
• All fidelity measures demonstrated parallel scales, providing consistent data across different scoring metrics.

Conclusions:

• Behavioral rehearsal is a reliable and valid method for assessing cognitive behavioral therapy adherence in pediatric populations, comparable to direct observation.
• Behavioral rehearsal offers a more efficient alternative to direct observation, potentially reducing resource demands in implementation research and practice.
• Findings support the use of behavioral rehearsal as a viable substitute for direct observation in evaluating CBT fidelity, particularly in resource-constrained settings.