Implications of question format in emergency department preventive health knowledge surveys

To determine if respondents differed by their demography (age group, race or ethnicity, and insurance status) in their tendency to correctly answer knowledge-based questions when they were in an agree-disagree instead of a multiple-choice format.

Women aged 18-55 years seeking treatment in the emergency department completed surveys of preventive health knowledge on three cancer screening tests (Pap smears, breast self-examinations, and mammograms) and two contraceptive measures (birth control pills and emergency contraception) that contained paired versions of questions in two different formats (agree-disagree format and multiple-choice format). Linear and ordinal regressions were used to determine demographic correlates of correctly answering the agree-disagree questions more often than the corresponding multiple-choice questions.

Of the 570 women included in this analysis, 64.6% were younger than 35 years, 62.1% were white, and 54.8% had private health care insurance. Older, white, and privately insured women demonstrated greater comprehension of all topics. Younger women, nonwhite women, and women without private health care insurance were more likely to respond to items correctly when they were in an agree-disagree format compared with a multiple-choice format.

This study demonstrated that survey responses are influenced by the format of the questions, particularly in certain demographic groups. Policy makers and researchers might draw false conclusions about the baseline knowledge and need for education of patients, especially in these populations. The use of agree-disagree format questions in preventive health knowledge surveys should be avoided whenever possible.