Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Experiment Videos

An examination of interrater reliability for scoring the Rorschach Comprehensive System in eight data sets.

Gregory J Meyer1, Mark J Hilsenroth, Dirk Baxter

  • 1Department of Psychology, University of Alaska, Anchorage 99508, USA. afgjm@uaa.alaska.edu

Journal of Personality Assessment
|June 18, 2002
PubMed
Summary

Related Concept Videos

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Evaluating LLM-Based Coders in Psychological Assessment: A Validation Framework With Application to the Rorschach Morbid Content Variable.

Assessment·2026
Same author

Relationship Between Personality Traits and Social Avoidance With Internet Gaming Disorder Severity.

Clinical psychology & psychotherapy·2026
Same author

Core principles of treating the suicidal adult: What we have learned from patients about restoring safety, emotion regulation, mentalizing, and epistemic trust.

Psychotherapy (Chicago, Ill.)·2025
Same author

Factors of treatment success in psychotherapy: a within-therapist analysis of early session processes.

Research in psychotherapy (Milano)·2025
Same author

Antisocial personality traits and outcome in psychotherapy: Does the therapeutic alliance mediate negative effects?

Psychotherapy (Chicago, Ill.)·2025
Same author

Behavioral signs of trauma on the Rorschach: Development of the Trauma Experience Index.

Psychological trauma : theory, research, practice and policy·2025

The Comprehensive System (CS) demonstrates excellent interrater reliability across various samples, confirming its robust scoring accuracy. Clinicians must maintain vigilance in scoring to ensure consistent and dependable results.

Area of Science:

  • Psychological assessment
  • Psychometrics
  • Forensic psychology

Background:

  • The Comprehensive System (CS) is a widely used method for scoring psychological tests.
  • Ensuring interrater reliability is crucial for the validity and consistency of assessment tools.

Purpose of the Study:

  • To evaluate the interrater reliability of the Comprehensive System (CS) across diverse samples.
  • To compare the reliability of CS summary scores versus individual response scores.
  • To validate methods for estimating response segment reliability within the CS.

Main Methods:

  • The study assessed interrater reliability in 8 distinct samples, including students, researchers, and clinicians.
  • Erroneous scores were systematically introduced into samples to test reliability under adverse conditions (10%, 20%, 30%).

Related Experiment Videos

  • Intraclass correlations were calculated to quantify reliability across different sample types and conditions.

    • Main Results:

    • Excellent interrater reliability was observed for 133 to 143 statistically stable CS scores, with median intraclass correlations ranging from .82 to .97.
    • CS summary scores exhibited higher reliability than scores for individual responses.
    • Reliability estimates were less stable in smaller samples, and Meyer's procedures for estimating response segment reliability were found to be accurate.

    Conclusions:

    • The Comprehensive System (CS) can be reliably scored, supporting its use in psychological assessment.
    • Consistent and accurate scoring relies on the skills and diligence of individual clinicians.
    • Ongoing monitoring of scoring accuracy by clinicians is essential for maintaining the integrity of CS assessments.