When Do Misunderstandings Matter? Evidence From Survey Interviews About Smoking

Area of Science:

• Social Sciences
• Communication Studies
• Survey Methodology

Background:

• Conceptual misalignments in dialogue can cause significant miscommunication.
• Mismeasurement in surveys by governments and social scientists carries substantial social costs.

Purpose of the Study:

• To investigate the conditions under which conceptual misalignments in dialogue lead to consequential miscommunication.
• To explore misunderstandings in survey interviews, particularly concerning ordinary expressions.

Main Methods:

• Quantified conceptual misalignment by probing respondent interpretations of survey terms in 131 tobacco use interviews.
• Re-administered survey questionnaires with standard definitions post-interview to assess impact.
• Analyzed instances where clarification attempts during interviews did or did not improve conceptual alignment.

Main Results:

• Respondent interpretations of terms like "smoking" and "every day" were highly variable and often misaligned with researcher intentions.
• 12% of respondents provided different answers on lifetime cigarette smoking after receiving a standard definition, indicating consequential mismeasurement.
• Clarification attempts did not consistently improve conceptual alignment; some failed or worsened understanding.

Conclusions:

• Conceptual misalignments are frequent in ordinary conversation and may go undetected.
• Attempts to detect and clarify misunderstandings in research settings are not always successful.
• Unresolved misunderstandings do not always impact survey outcomes, but their potential for mismeasurement is significant.