Comparing Cognitive Interviewing and Psychometric Methods to Evaluate a Racial/Ethnic Discrimination Scale

Area of Science:

• Social Sciences
• Psychology
• Survey Methodology

Background:

• Survey evaluation often recommends combining qualitative and quantitative methods, but this is rarely implemented.
• The Everyday Discrimination scale (EDS) measures discrimination frequency across diverse populations.

Purpose of the Study:

• To evaluate the Everyday Discrimination scale (EDS) using integrated qualitative and quantitative methodologies.
• To identify and address potential biases in the EDS for multi-ethnic populations.

Main Methods:

• Qualitative: Cognitive interviewing with 30 diverse participants to identify item clarity and consistency issues.
• Quantitative: Psychometric analysis of secondary data from national studies (N=4400+) including Asian Americans, Latinos, and African Americans.
• Integrated approach: Used qualitative findings to contextualize quantitative results, including differential item functioning (DIF) by race/ethnicity.

Main Results:

• Cognitive testing revealed redundant, unclear, or inconsistent items within the EDS.
• Psychometric analysis identified redundant items and DIF by race/ethnicity.
• Qualitative and quantitative methods provided complementary insights into scale performance.

Conclusions:

• Integrated mixed-methods evaluation effectively identified and explained issues with the EDS.
• Researchers should adopt combined qualitative and quantitative approaches in instrument pretesting.
• This integration can minimize response bias among ethnic and racial respondents in population-based surveys.