Sir Francis Galton and proficiency testing in cytopathology

Data from the National Cytology Proficiency Testing Update show that as of January 31, 2006, 9% of 12,786 examinees failed the test on the first attempt. For the second attempt, the failure rate among those who had initially failed remained surprisingly similar, 10%, although common sense would dictate that it should be much higher among those who have already failed the test once and should have lower professional skills. What is the reason for this remarkable improvement in performance? There is a simple explanation: this is a statistical phenomenon, known as "regression toward the mean." Two groups of examinees earn failing scores during proficiency testing: those whose skills are really insufficient and those who are competent but who achieved lower scores due to random variation in the test results. The latter, "misclassified" examinees subsequently "regress" toward the mean during the second test; that is, their test results become more commensurate with their genuine skills. Since the failure rates of all participants during the first and second attempts were similar, we must assume that the majority of the examinees who failed on the first attempt fall into the second, misclassified group, and only a minority have truly insufficient skills.