The effect of collapsing multinomial data when assessing agreement.

Area of Science:

• Epidemiology
• Biostatistics

Background:

• Researchers often use proxy informants in epidemiological studies when primary subjects are unavailable.
• Formal statistical inference to assess agreement between proxy informants and primary subjects is underutilized.
• This study focuses on interobserver agreement with two raters and multiple outcome categories.

Purpose of the Study:

• To evaluate the impact of dichotomizing multinomial data on confidence interval width for the kappa coefficient.
• To assess how dichotomization affects sample size requirements for hypothesis testing concerning kappa.
• To compare inference procedures for interobserver agreement with multinomial versus dichotomous data.

Main Methods:

• Simulation studies were employed to compare confidence interval coverage and width.
• Sample size requirements were compared for both multinomial and dichotomous data scenarios.
• The study utilized a published dataset on drinking habits involving primary and proxy respondents.

Main Results:

• Treating multinomial data as dichotomous significantly increases expected confidence interval widths.
• Dichotomization leads to severe penalties in sample size requirements for hypothesis testing.
• The observed effects were demonstrated using a real-world dataset.

Conclusions:

• There are significant advantages to maintaining multinomial data in its original form.
• Collapsing data into a binary trait can lead to loss of statistical power and efficiency.
• Preserving the original scale of multinomial data is recommended for robust agreement analysis.