Expanding Integrity Measurement in Mental Health Care: A Coding System of Targets, Practices, and Steps

Area of Science:

• Mental Health Services Research
• Healthcare Quality Improvement
• Implementation Science

Background:

• Service integrity, comparing delivered care to expectations, is vital for managing service quality.
• Existing metrics may not fully capture the nuances of service delivery in dynamic settings.
• Youth mental health systems face challenges with service inequities, necessitating robust quality assessment tools.

Purpose of the Study:

• To test the reliability of an expanded set of service integrity metrics.
• To evaluate metrics including targets, practices, and steps at different dialogue levels and service event types.
• To inform the development of more comprehensive service quality assessments.

Main Methods:

• The study was conducted in two under-resourced, publicly funded youth mental health systems.
• Targets, practices, and steps were coded in 78 treatment and 84 clinical supervision events.
• Interrater reliability was estimated using Fleiss' kappa and intraclass correlation coefficients.

Main Results:

• Coders demonstrated acceptable reliability in detecting targets, practices, and steps during supervision and treatment.
• Steps were detected more reliably than practices; targets were more reliable in excerpts than complete events.
• Supervision content was coded more reliably than treatment content.

Conclusions:

• A uniform set of metrics can be reliably coded to expand the analysis of service integrity in dynamic care settings.
• Assessing delivered practices against targets and supervision provides crucial insights for improving care.
• Findings support enhanced service coordination, continuity, and overall quality in mental health care.