Issues in the interpretation and reporting of surveys in dental education

Surveys are the most common form of data-based article published in the Journal of Dental Education. The apparent ease with which they are conducted and the seeming simplicity of reporting results mask significant issues in sample design and performing maximally useful analyses. Four concerns are discussed here. First, it is demonstrated that results are a function of who, when, and where responses are sampled, each source making independent contributions. Second, absolute sample size is shown to be the most significant factor affecting precision in surveys, and the numbers of schools, respondents, and other sources of variance can be chosen to minimize survey imprecision. Third, response rate typically has negligible effect on precision and an uncertain effect on accuracy (freedom from bias). A technique, sample saturation, is explained that can be used to protect, to some degree, surveys from the effects of bias. Finally, suggestions are offered for reporting survey results in a visually meaningful fashion, and an appeal is made that recommendations associated with surveys not be published unless they are grounded in both data and well-developed theory. This analysis references a previously published survey on competency-based dental education to illustrate methodological points in concrete terms.