A comparison of self-reported and phone-administered methods of ACT fidelity assessment: a pilot study in Indiana

Monitoring fidelity of assertive community treatment (ACT) teams is costly. This study investigated the reliability and validity of a less burdensome approach: self-reported assessment.

Phone-administered and self-reported assessments were compared for 16 ACT teams. Team leaders completed a self-report protocol providing information sufficient to score the Dartmouth Assertive Community Treatment Scale (DACTS). Two raters scored the DACTS using only self-reported information. Two additional raters conducted phone interviews with team leaders, verifying the self-reported data, and independently scored the DACTS.

DACTS total scores obtained via self-reported assessments were reliable and valid compared with phone-administered assessment on the basis of interrater consistency (intraclass correlation) and consensus (mean rating differences). Phone-administered assessments agreed with self-reported assessments within .25 scale points (out of 5 points) for 15 of 16 teams.

A self-report approach could address concerns regarding costs of monitoring as part of a stepped approach to quality assurance.