Related Experiment Videos
Maximum likelihood estimation of the kappa coefficient from models of matched binary responses
M M Shoukri1, S W Martin, I U Mian
1Department of Population Medicine, University of Guelph, Ontario, Canada.
Statistics in Medicine
|January 15, 1995
Summary
This study estimates the kappa-coefficient of agreement for binary ratings, finding it depends on intraclass correlation. We also analyzed the statistical power and estimator properties for this agreement measure.
Area of Science:
- Statistics
- Biostatistics
- Psychometrics
Background:
- Inter-rater reliability is crucial for consistent data collection.
- The kappa-coefficient is a common metric for assessing agreement between raters.
- Understanding factors influencing kappa is essential for accurate interpretation.
Purpose of the Study:
- To estimate the kappa-coefficient of agreement for binary ratings.
- To investigate the dependence of kappa on the intraclass correlation coefficient.
- To evaluate the statistical power and estimator properties of kappa.
Main Methods:
- Utilized matched pairs of binary responses.
- Employed Monte Carlo simulation to assess statistical power.
- Analyzed the bias and variance of the maximum likelihood estimator for kappa.
Main Results:
- The kappa-coefficient estimate is shown to depend on the common intraclass correlation coefficient.
- Investigated the power of the significance test for kappa.
- Examined the large sample bias and variance of the maximum likelihood estimator.
Conclusions:
- The intraclass correlation coefficient is a key determinant of the kappa-coefficient of agreement.
- The study provides insights into the statistical performance of kappa and its estimators.
- Findings are relevant for researchers using kappa to assess inter-rater reliability.