Observations on some foul papers on medical education. a referee's whistle

Faults commonly seen in papers on medical education in general and examination techniques in particular include: (a) Inadequate replication and neglect of the novelty effect on the results of experiments, (b) excessive reliance on internal analysis in assessing examination techniques, without seeking external evidence that the right candidates are being passed, (c) uncritical acceptance of the doctrine of a necessary core of knowledge and of pass levels derived from prior reasoning, (d) assumption that because recall and recognition methods produce different pass levels the one must necessarily be "better" than the other. It is suggested that good examiners look not for specific items of knowledge, but for evidence of a sufficient total quantity combined with sufficient judgment in the choice of what to learn, and that the one common essential attribute of good medical students and good doctors is an exceptionally high non-specific capacity for information processing.