[Comparison of missing data handling methods for AC₁ coefficient estimation]

To explore the impact of different missing data handling methods on AC₁ coefficient estimation through simulation studies.

Monte Carlo simulation was used to generate evaluation data under different missing mechanisms. The parameters generated included the number of raters, categories, sample size, disease prevalence, random rating probability, and missing proportion. Four missing data handling methods, by excluding subjects with zero ratings, excluding subjects with incomplete ratings, rater mode imputation, and subject mode imputation, were compared using bias and mean squared error (MSE) as metrics.

When disease prevalence was balanced or the missing data mechanism was missing completely at random (MCAR) or at random (MAR), excluding subjects with zero ratings showed the best performance, with bias and MSE close to zero at a missing proportion below 30%. Under skewed prevalence and missing not at random (MNAR), subject mode imputation was superior for AC₁ coefficient estimation, resulting in a bias within ±0.10 and an MSE below 0.09; for a sufficient sample size and a missing proportion ≤30%, the MSE of this method was nearly zero. Rater mode imputation showed the worst performance across all these scenarios. Excluding subjects with incomplete ratings resulted in an acceptable error only in relatively simple settings (two raters and two categories) with low a missing proportion under MCAR/MAR, but showed a poor stability in other scenarios.

No universally optimal method exists for handling missing data in AC₁ estimation. We recommend excluding subjects with zero ratings for balanced prevalence or MCAR/MAR, and subject mode imputation for skewed prevalence under MNAR. Researchers should report AC₁ estimates from multiple methods to allow assessment of result sensitivity.