Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Experiment Videos

Agreement measurement and the judgment process.

C L Janes

    The Journal of Nervous and Mental Disease
    |June 1, 1979
    PubMed
    Summary
    This summary is machine-generated.

    Correction for chance agreement between raters is not always necessary. When needed, Maxwell's Random Error Coefficient of Agreement (RE) is recommended over kappa (K) due to its logical accord with judgment processes.

    Related Concept Videos

    You might also read

    Related Articles

    Articles linked to this work by shared authors, journal, and citation graph.

    Sort by
    Same author

    Problem boys in young adulthood: Teachers' ratings and twelve-year follow-up.

    Journal of youth and adolescence·2013
    Same author

    Using focus groups to assess primary care patients' satisfaction.

    Evaluation & the health professions·2000
    Same author

    The health and health care utilization of people in Buncombe County.

    North Carolina medical journal·1997
    Same author

    The Claxton Elementary School Health Program: merging perceptions and behaviors to identify problems.

    The Journal of school health·1995
    Same author

    The effect of stimulus significance on skin conductance recovery.

    Psychophysiology·1985
    Same author

    Intelligence, classroom behavior, and academic achievement in children at high and low risk for psychopathology: a structural equation analysis.

    Journal of abnormal child psychology·1984

    Area of Science:

    • Psychometrics
    • Statistical analysis
    • Inter-rater reliability

    Background:

    • Assessing agreement between two raters is crucial in many fields.
    • Chance agreement can inflate observed reliability, necessitating correction.
    • Existing measures like kappa (K) have theoretical limitations.

    Purpose of the Study:

    • To review the rationale for chance correction in inter-rater agreement.
    • To compare kappa (K) and Maxwell's Random Error Coefficient of Agreement (RE).
    • To provide recommendations for appropriate agreement measures.

    Main Methods:

    • Literature review of chance correction methodologies.
    • Comparative analysis of statistical assumptions underlying kappa (K) and Maxwell's RE.

    Related Experiment Videos

  • Examination of how judgment data are processed by each measure.

    • Main Results:

    • Chance correction is not universally appropriate or necessary.
    • Kappa (K) has questionable assumptions regarding the judgment process.
    • Maxwell's RE aligns better with how judgment data are constructed.

    Conclusions:

    • Avoid chance correction when inappropriate.
    • Kappa (K) use is discouraged due to methodological incompatibility.
    • Maxwell's RE is recommended when disagreement cells have equal proportions.