Performance of a Supervisor Observational Coding System and an Audit and Feedback Intervention.

Area of Science:

• Mental Health Services Research
• Clinical Psychology
• Implementation Science

Background:

• Workplace-based clinical supervision is crucial for delivering evidence-based treatments (EBTs) in community mental health care for youth.
• Effective methods are needed to monitor and support supervisor performance without disrupting workflow.
• Audit and Feedback (A&F) interventions are a potential strategy for enhancing supervision quality.

Purpose of the Study:

• To psychometrically evaluate an observational coding system for measuring adherence and competence in EBT supervision.
• To assess the impact of a Supervisor Audit and Feedback (SAF) intervention on supervisor adherence and competence.

Main Methods:

• A 7-month randomized controlled trial involving 60 clinical supervisors.
• Supervisors recorded weekly sessions; the experimental group received monthly web-based feedback reports.
• Psychometric evaluation used Item Response Theory; intervention effects were analyzed with mixed-effects regression models.

Main Results:

• The observational coding system demonstrated good psychometric properties, though coder reliability was lower for competence than adherence.
• The SAF intervention showed statistically significant positive effects on supervisor adherence and, to a lesser extent, competence.
• Monthly electronic feedback reports were promising for maintaining or improving supervisor performance.

Conclusions:

• The developed observational coding system is a viable tool for measuring EBT supervision.
• Supervisor Audit and Feedback interventions can effectively support and enhance clinical supervisor performance in community-based settings.
• Further research is needed to assess generalizability to other EBTs and populations.