Performance appraisal reactions: measurement, modeling, and method bias

Area of Science:

• Organizational Psychology
• Human Resource Management
• Psychometrics

Background:

• Performance appraisal reactions are crucial for employee attitudes and behaviors.
• Existing scales for measuring these reactions require rigorous validation.
• Understanding measurement properties is key to accurate assessment.

Purpose of the Study:

• To comprehensively examine the measurement of performance appraisal reactions.
• To assess the construct validity of commonly used reaction scales.
• To investigate the factor structure and potential biases in appraisal reaction data.

Main Methods:

• Confirmatory factor analysis (CFA) was employed to evaluate scale measurement.
• Higher-order and nested factor models were compared for fit.
• Techniques by Williams et al. (1996) were used to test for affective method biases.

Main Results:

• Reaction scales demonstrated favorable measurement of appraisal reactions, with minor concerns.
• A higher-order appraisal reactions model provided a good fit to the data.
• A single-general-factor model did not adequately fit the data.
• Neither positive nor negative affect significantly biased reaction measures at measurement or construct levels.

Conclusions:

• Current performance appraisal reaction scales are largely valid and reliable.
• A higher-order factor structure best represents appraisal reactions.
• Self-report data on appraisal reactions are not significantly driven by affective states, indicating robust measurement.