Combining Content Validation and Cognitive Interviewing: A Methodological Approach for New Instrument Development

Critiques of traditional content validation is its limitation to a positivist approach to quantifying a question's relevance while cognitive interviewing is labor and resource-intensive. The purpose of this study is to describe a novel approach to developing a new survey instrument that combines content validity indexing and cognitive interviewing to address the limitations of both approaches. Guided by language planning theory for health services research, the a priori target constructs for instrument development included: (1) organizational language access resources and policies; (2) nursing workflow and decision-making related to interpreter use; and (3) perceived quality and outcomes of care in the presence of language barriers. Content validation with 50 expert raters was the first step, followed by four recorded qualitative cognitive interviews focused on poorly performing items. Transcripts were analyzed using a qualitative descriptive approach. Scores revealed items where raters lacked consensus. Limiting cognitive interviewing to these items enabled rich discussions of why items were potentially problematic, improved phrasing, and determined the optimal scaling. Three new survey instruments resulted from the process. The approach reduced instrument development time and proved to be highly complementary. Replication of the approach is recommended to further refine and test it.