A Measurement Gap? Effect of Survey Instrument and Scoring on the Partisan Knowledge Gap

Area of Science:

• Political Science
• Survey Methodology
• Psychology

Background:

• Partisan gaps in political knowledge are significant in the U.S.
• Commercial surveys may overestimate these gaps due to survey design.
• Motivated guessing versus knowledgeable "cheerleading" are debated explanations.

Purpose of the Study:

• To differentiate between confidently held beliefs and motivated guessing in partisan knowledge gaps.
• To assess the impact of survey features on the measurement of political knowledge.
• To determine the true extent of partisan differences in political knowledge.

Main Methods:

• Conducted experiments manipulating knowledge item features in commercial surveys.
• Compared partisan knowledge gaps using standard versus modified survey scales.
• Analyzed data to identify sources of upward bias in reported knowledge.

Main Results:

• Survey items with guessing-encouraging features inflate partisan knowledge gaps.
• Removing these features reduced partisan gaps by approximately 40%.
• Motivated guessing by less knowledgeable partisans, not "cheerleading," explains the bias.

Conclusions:

• Commercial survey data likely overestimate partisan differences in political knowledge.
• Survey design significantly impacts the measurement of political knowledge and partisan divides.
• A substantial portion of reported partisan knowledge gaps stems from methodological artifacts.