A Cross-Classified CFA-MTMM Model for Structurally Different and Nonindependent Interchangeable Methods

Area of Science:

• Psychology
• Quantitative Psychology
• Statistical Modeling

Background:

• Multirater (multimethod, multisource) studies are common in psychology.
• Existing multilevel confirmatory factor models (ML-CFA-MTMM) assume independent raters.
• Real-world data often violate this independence, creating cross-classified structures.

Purpose of the Study:

• To extend the ML-CFA-MTMM model for cross-classified multirater designs.
• To develop a model that accounts for nonindependent interchangeable raters.
• To explicitly model the interaction between targets and raters as a latent variable.

Main Methods:

• Introduction of the C4 model (Cross-Classified CTC[M-1] Combination of Methods).
• Application of Bayesian estimation techniques.
• Utilizing a real data application for validation.

Main Results:

• The C4 model successfully accounts for nonindependent interchangeable raters in MTMM data.
• Demonstration of obtaining credibility intervals for model parameters and variance components.
• Validation of the model's utility in a real data application.

Conclusions:

• The C4 model provides a robust framework for analyzing complex multirater data in psychology.
• This approach enhances the accuracy of statistical modeling when rater independence is violated.
• Bayesian estimation facilitates credible parameter and variance component estimation in these designs.