Combining cognitive and statistical approaches to survey design

Sample surveys provide data for academic research, government policy-making, the media, and business. Statistical research aims to improve survey data by reducing extraneous sources of variability and thus increasing accuracy. Researchers have begun to use paradigms adapted from the cognitive sciences to study those sources of variability associated with the processes that the respondent undertakes in understanding questions, remembering, judging and estimating, and formulating answers. To generalize laboratory-based findings, researchers must begin to embed designed experiments that vary the questionnaire content into sample surveys of broad population. Issues associated with the design of and statistical inference from such embedded experiments are examined and illustrated with an example on the effects of context questions on responses in attitude surveys.