Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Experiment Videos

Exact and resampling probability values for weighted kappa.

Kenneth J Berry1, Janis E Johnston, Paul W Mielke

  • 1Department of Sociology, Colorado State University, Fort Collins, CO 80523-1784, USA. berry@lamar.colostate.edu

Psychological Reports
|June 9, 2005
PubMed
Summary
This summary is machine-generated.

Related Concept Videos

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Spatial-temporal association of soil Pb and children's blood Pb in the Detroit Tri-County Area of Michigan (USA).

Environmental research·2020
Same author

The concurrent decline of soil lead and children's blood lead in New Orleans.

Proceedings of the National Academy of Sciences of the United States of America·2019
Same author

Spatiotemporal exposome dynamics of soil lead and children's blood lead pre- and ten years post-Hurricane Katrina: Lead and other metals on public and private properties in the city of New Orleans, Louisiana, U.S.A.

Environmental research·2017
Same author

Spatiotemporal dynamic transformations of soil lead and children's blood lead ten years after Hurricane Katrina: New grounds for primary prevention.

Environment international·2016
Same author

Evolving from reactive to proactive medicine: community lead (Pb) and clinical disparities in pre- and post-Katrina New Orleans.

International journal of environmental research and public health·2014
Same author

Environmental and health disparities in residential communities of New Orleans: the need for soil lead intervention to advance primary prevention.

Environment international·2012
Same journal

A Scoping Review of Nonverbal Synchrony Between Individuals With Social Anxiety and Others.

Psychological reports·2026
Same journal

Finding Freudenfreude: Deriving Subjective Well-Being From Passive Observation of a Relational Tie's Happiness via Social Media Post.

Psychological reports·2026
Same journal

A Cross-Sectional Study Comparing Flourishing and Quality of Life as Indicators of Psychological Well-Being in Adults.

Psychological reports·2026
Same journal

Understanding Love in Couple Relationships: A Scoping Review of Sternberg's Triangular Theory.

Psychological reports·2026
Same journal

Psychological Impacts of Instagram Use: The Interplay of Social Comparison, Self-Esteem, and Anxiety.

Psychological reports·2026
Same journal

The Effect of Self-Compassion on Shame in Post-Event Processing.

Psychological reports·2026
See all related articles

Exact and resampling permutation procedures offer accurate probability values for weighted kappa. Resampling methods are suitable for all data types, while exact methods excel with sparse datasets.

Area of Science:

  • Statistics
  • Biostatistics
  • Psychometrics

Background:

  • Weighted kappa is a crucial metric for inter-rater reliability.
  • Accurate probability values are essential for hypothesis testing with weighted kappa.
  • Traditional asymptotic methods may be unreliable with sparse data.

Purpose of the Study:

  • To describe permutation procedures for calculating exact and resampling probability values for weighted kappa.
  • To compare the performance of exact and resampling permutation procedures against asymptotic methods.

Main Methods:

  • Developed exact permutation procedures for weighted kappa probability values.
  • Developed resampling permutation procedures for weighted kappa probability values.
  • Compared permutation-based probabilities with asymptotic probabilities.

Related Experiment Videos

Main Results:

  • Exact permutation procedures provide accurate probabilities for sparse datasets.
  • Resampling permutation procedures offer accurate probabilities for both sparse and nonsparse datasets.
  • Permutation methods demonstrate advantages over asymptotic methods, especially with limited data.

Conclusions:

  • Permutation procedures are valuable for computing accurate weighted kappa probabilities.
  • Resampling permutation methods offer a robust approach for diverse data sparsity levels.
  • These methods enhance the reliability of statistical inference in inter-rater reliability studies.