To Be or Not to Be: Exploring the Nature of Positively and Negatively Keyed Personality Items in High-Stakes Testing

Including equal numbers of positively and negatively keyed items is common in Five-Factor Model (FFM) personality measures. Much literature has demonstrated the presence of positive and negative keying factors in low-stakes testing situations, but there is a dearth of research investigating these factors in high-stakes testing. To address this gap, we investigated whether an FFM measure used in high-stakes testing was influenced by positive and negative keying factors. We also examined the overlap of the positive and negative keying factors with social desirability, rule-consciousness, acquiescence, and cognitive ability. Confirmatory factor analysis supported the inclusion of distinct factors associated with positively and negatively keyed items and suggested that the keying factors accounted for a substantial portion of variation in responses to FFM items. Social desirability and rule-consciousness were found to have significant relations with both keying factors, whereas acquiescence was only related to the negative keying factor. Implications for the construct validity of FFM measures used in high-stakes testing and directions for future research are discussed.