Empirical support for interrater reliability of Rorschach Comprehensive System coding

Although a great deal of data has been published in the past 20 years supporting the interrater reliability of the Rorschach, recently commentators have raised anew concerns over the interrater reliability of this well-known and frequently used measure. An analysis of the literature reveals that these concerns are based on a selective review of the literature. The current study reports interrater reliability statistics for 70 interpretive Rorschach variables (n = 84), breaking down intraclass correlations statistics by base rate. Results confirm the strong empirical evidence of the interrater reliability of this measure when scored by a well-trained and diverse group of researchers and clinicians. Reliability is especially strong and consistent for the high base-rate variables from which clinicians often base their interpretations. These data further suggest that large samples are needed for stable reliability estimates of low base-rate variables and that estimates of the reliability of low base-rate variables are subject to error.