The mystique of the mistake. With proposed standards for validating proficiency tests in anatomic pathology

Variability in classification in anatomic pathology does not necessarily indicate that a mistake has been made. It is usually an artifact, created when pathologists choose a single category from among two or more justifiable alternatives. This is most common when standard classifications with uniform terminology are not used. It also can occur when classification systems are not constructed so as to insure mutual exclusivity of categories. It is proposed that a proficiency test in anatomic pathology should not be considered scientifically valid until a professional organization primarily concerned with anatomic pathology has endorsed its proposed classification system as having categories that are close to 100% mutually exclusive in the hands of expert pathologists not involved in developing the system. All possible precautions should be taken to insure that the "right answers" for any proficiency test are generated in a way that excludes the possibility of multiple justifiable alternatives.