A Note on the Linearly and Quadratically Weighted Kappa Coefficients

Area of Science:

• Statistics
• Psychometrics
• Data Analysis

Background:

• Inter-rater agreement is crucial for reliable ordinal scale measurements.
• Linear and quadratic weighted kappa coefficients are widely used for this purpose.
• Previous work highlighted specific properties of these kappa coefficients.

Purpose of the Study:

• To present a rank-one matrix decomposition approach for weighted kappa statistics.
• To demonstrate the properties of linearly and quadratically weighted kappa coefficients using this novel method.
• To provide a concise mathematical framework for understanding these agreement measures.

Main Methods:

• The study employs a rank-one matrix decomposition technique.
• This approach is applied to the weighting schemes of kappa coefficients.
• The method is used to analyze embedded 2 by 2 agreement matrices.

Main Results:

• The linear weighted kappa is shown to be a weighted average of embedded 2 by 2 kappa coefficients.
• The quadratic weighted kappa is demonstrated to be invariant to row or column symmetric agreement matrices.
• The rank-one decomposition provides a unified and concise explanation for these phenomena.

Conclusions:

• Rank-one matrix decomposition offers an elegant method for analyzing weighted kappa statistics.
• This approach clarifies the distinct behaviors of linear and quadratic weighting schemes.
• The findings enhance the understanding of inter-rater reliability measures on ordinal scales.