Interrater reliability of using brief standardized outcome measures in a community mental health setting

Given psychiatry's need to implement measurement-based care, the study examined whether direct-care staff could reliably administer brief positive and negative symptom instruments to track symptom changes and inform clinical decision making.

Raters (82 case managers) were assessed at baseline. Training was provided for individuals not meeting reliability criteria. These individuals were reassessed to determine the effect of training. In addition, rater drift was assessed for raters judged to be reliable at baseline.

Seventy-seven percent of direct-care staff met criteria for reliability either at baseline or after they received additional training.

A majority of direct-care staff can be trained to reliability on brief scales of positive and negative symptoms that can be used to guide clinical decision making.