Rater agreement and the generalized Rudas-Clogg-Lindsay index of fit

Area of Science:

• Statistics
• Social Sciences
• Psychometrics

Background:

• Contingency table analysis is crucial for statistical modeling.
• Assessing agreement between raters is a common challenge.
• Existing fit indices may have limitations with complex models or data sparsity.

Purpose of the Study:

• To introduce and evaluate a generalized RCL index of fit for contingency table analysis.
• To extend the application of the RCL index to assess inter-rater agreement.
• To demonstrate the index's utility with complex models and sparse data.

Main Methods:

• Proposed a two-component mixture model for contingency tables.
• Defined the RCL index of lack of fit based on a minimal mixing weight (w*).
• Applied the generalized RCL index to assess agreement under pure agreement, quasi-independence, and independence hypotheses.
• Utilized linear logistic latent class analysis for parameter estimation.

Main Results:

• The generalized RCL index effectively assesses fit in contingency tables, even with empty cells.
• The method demonstrated successful application in 3x3 and 4x4 tables with varying cell frequencies.
• Parameter estimation and determination of w* were feasible within the linear logistic framework.
• The approach accommodates models beyond standard contingency table families, including latent class models.

Conclusions:

• The number of model components can exceed one, supporting the generalized RCL index.
• The RCL index of fit can be extended to more complex statistical models.
• The presence of empty cells does not impede the application of this index, making it suitable for small and large sample sizes.