A non-iterative confidence interval estimating procedure for the intraclass kappa statistic with multinomial outcomes

Area of Science:

• Statistics
• Biostatistics
• Medical Research Methodology

Background:

• The intraclass kappa statistic is crucial for assessing inter-rater reliability, particularly with multiple outcome categories.
• Accurate confidence intervals are essential for interpreting the reliability estimates derived from the kappa statistic.
• Existing methods for confidence interval construction can be computationally intensive or less accurate for smaller sample sizes.

Purpose of the Study:

• To derive the asymptotic sample variance of the intraclass kappa statistic for multinomial outcome data.
• To develop a non-iterative, modified Wald type procedure for constructing confidence intervals.
• To evaluate the performance of the proposed method through a simulation study.

Main Methods:

• Derivation of the asymptotic sample variance for the intraclass kappa statistic.
• Application of a modified Wald type procedure for confidence interval estimation.
• Simulation study to assess confidence interval coverage and width.

Main Results:

• The proposed method provides accurate confidence interval coverage and width.
• Effective performance was observed even for sample sizes as small as 50.
• The non-iterative approach simplifies the calculation of confidence intervals.

Conclusions:

• The developed statistical procedure offers a reliable and efficient method for constructing confidence intervals for the intraclass kappa statistic.
• This approach is particularly valuable in medical research where multinomial data and smaller sample sizes are common.
• The method enhances the interpretability of inter-rater reliability in various scientific applications.