Analysis of Multitrait-Multirater Performance Appraisal Data: Composite Direct Product Method versus Confirmatory...

Area of Science:

• Psychology
• Organizational Behavior
• Quantitative Psychology

Background:

• Multitrait-multirater (MTMR) performance appraisal data involves assessing multiple traits by multiple raters.
• Understanding how trait variance and rater variance interact is crucial for accurate performance evaluations.
• Existing models, like Confirmatory Factor Analytic (CFA), often assume additive relationships, which may not fully capture complex interactions.

Purpose of the Study:

• To investigate the combination of trait and rater variance in MTMR data.
• To compare the additive assumption of the CFA model with the multiplicative assumption of the Composite Direct Product (CDP) model.
• To determine which model provides a superior fit for empirical MTMR data.

Main Methods:

• Utilized Confirmatory Factor Analytic (CFA) and Composite Direct Product (CDP) models.
• Empirically tested both models using four distinct datasets of MTMR performance appraisal data.
• Analyzed the fit of each model to assess the relationship between trait and rater variance.

Main Results:

• The Composite Direct Product (CDP) model demonstrated a superior fit compared to the Confirmatory Factor Analytic (CFA) model across all four datasets.
• Results indicate that a multiplicative relationship between trait and rater variance provides a more accurate representation of MTMR data.
• The study highlights empirical differences supporting the CDP model's efficacy.

Conclusions:

• The CDP model offers a more realistic representation of trait and rater variance interactions in MTMR data than the CFA model.
• The findings challenge the simplicity of additive assumptions, emphasizing the benefits of multiplicative models for complex performance appraisal data.
• Guidelines for applying the CDP method are provided to enhance the analysis of MTMR data.