The reporting of statistics in medical educational studies: an observational study

There is confusion in the medical literature as to whether statistics should be reported in survey studies that query an entire population, as is often done in educational studies. Our objective was to determine how often statistical tests have been reported in such articles in two prominent journals that publish these types of studies.

For this observational study, we used electronic searching to identify all survey studies published in Academic Medicine and the Journal of General Internal Medicine in which an entire population was studied. We tallied whether inferential statistics were used and whether p-values were reported.

Eighty-four articles were found: 62 in Academic Medicine and 22 in the Journal of General Internal Medicine. Overall, 38 (45%) of the articles reported or stated that they calculated statistics: 35% in Academic Medicine and 73% in the Journal of General Internal Medicine.

Educational enumeration surveys frequently report statistical tests. Until a better case can be made for doing so, a simple rule can be proffered to researchers. When studying an entire population (e.g., all program directors, all deans, and all medical schools) for factual information, do not perform statistical tests. Reporting percentages is sufficient and proper.