Validity of Clinician's Self-Reported Practice Elements on the Monthly Treatment and Progress Summary

Area of Science:

• Mental Health Services Research
• Clinical Psychology
• Health Informatics

Background:

• Growing demand for accountability in community mental health systems necessitates reliable measures of therapeutic practice.
• The Monthly Treatment and Progress Summary (MTPS) is a clinician-report tool for assessing therapeutic strategies, targets, and progress.
• Evaluating the validity of the MTPS is crucial for its effective implementation in service systems.

Purpose of the Study:

• To assess the validity of therapeutic strategies documented in the MTPS.
• To compare clinician-reported practices with independent coder assessments.
• To refine the MTPS coding system for enhanced reliability and accuracy.

Main Methods:

• Utilized 47 audio recordings from 19 youth therapy sessions for analysis.
• Trained observers reliably coded 12 discrete therapeutic practices.
• Employed clinician-coder agreement, including intraclass correlations, to establish validity.

Main Results:

• Initially, 4 of 12 practices demonstrated validity based on clinician-coder agreement (ICC ≥0.60).
• A revised coding system, incorporating extensiveness and experiential scales, achieved 100% validity.
• The study confirmed the reliability of the revised MTPS in capturing therapeutic practices.

Conclusions:

• The revised MTPS coding system provides a valid and reliable measure of therapeutic practices in community mental health.
• The MTPS has significant implications for improving accountability and quality of care in mental health service systems.
• Further research is recommended to explore the broader applications and long-term impact of the MTPS.