Item generation and establishing face and content validity of a rating scale: A primer

This article is the third in a series of learning resources on translation/adaptation of a scale, new scale development and testing its psychometric properties. This paper deals with the process of domain identification and item generation for a new scale and how to perform content validity (CV). During the scale development, the domain of interest/construct must be well-articulated and decided a priori. Item generation should be informed by deductive as well as inductive methods to capture a comprehensive item pool. Items for a scale should be simple, unambiguous, accounting for the experiences of the target population and follow a normal conversation. CV assesses the relevance and representativeness of a scale's items. It is assessed through face validity (involving the target population), content validity ratio (CVR), content validity index (item and scale level), and interrater agreement (percent agreement, Cohen's kappa, and Fleiss' generalized kappa) by involving experts. In this paper, we provide insights and practical guidance on how to generate items for a scale and perform content validity checks. We have provided, and practical guide for conducting analyses and interpreting results on SPSS and MS Excel for analysis.