Psychometric properties of a decision-support tool for the era of managed care

This article reports psychometric properties of a decision-support scale designed to quantify the decision-making process for allocating psychiatric care. The authors developed a scale to evaluate the level of care needed for patients requiring psychiatric treatment in a health maintenance organization (HMO) setting. This study examines the reliability and validity of that scale by measuring interrater agreement among utilization reviewers from the HMO and between those reviewers and the clinicians who evaluated the patients directly. Agreement (kappa) among the five clinical raters acting in a utilization review capacity on dimensions of the scale ranged from .71 to .98. Kappas for agreement between the treatment intensity proposed by the reviewer/rater and used by the treating clinician were .41, .38, .40, .35, .36, and .39, respectively. The scale is reliable in the hands of trained personnel. Although there were important differences between clinicians' ratings and those of utilization review raters, these differences do not suggest that use of the scale would limit patient access to care.