Combining cognitive and statistical approaches to survey design.

Area of Science:

• Cognitive science
• Survey methodology
• Statistical inference

Background:

• Sample surveys are crucial for research, policy, media, and business.
• Statistical research enhances survey data accuracy by minimizing variability.
• Cognitive science paradigms are increasingly used to understand respondent survey response variability.

Purpose of the Study:

• To explore the design and statistical inference of embedded experiments within sample surveys.
• To generalize laboratory findings on cognitive processes in surveys to broader populations.
• To investigate the impact of context questions on attitude survey responses.

Main Methods:

• Embedding designed experiments that vary questionnaire content into large-scale sample surveys.
• Applying cognitive science principles to analyze sources of variability in survey responses.
• Examining statistical inference methods for embedded experimental designs.

Main Results:

• Embedded experiments are feasible for generalizing laboratory findings.
• Context questions can significantly influence responses in attitude surveys.
• Specific methods for designing and analyzing embedded experiments were illustrated.

Conclusions:

• Integrating cognitive science and survey methodology through embedded experiments improves data quality.
• Careful design and statistical analysis are essential for valid inference from embedded experiments.
• Understanding context effects is vital for accurate attitude measurement in surveys.