Changes in participant performance in the "test-taking" environment: observations from the 2006 College of American...

Because the consequences of making an interpretive error on a proficiency test are more severe than those made on an educational challenge, the same slide may exhibit different performance characteristics in the 2 different settings.

The results of the 2006 College of American Pathologists Gynecologic Cytology Proficiency Testing Program (PAP PT) provide the opportunity to compare the performance characteristics of the field-validated slides in the PAP PT environment with those of the same graded slides in the College of American Pathologists Educational Program (formerly known as the PAP Program).

All participant responses for negative (category B) and positive (categories C and D) validated slides in the 2006 PAP PT were used to determine the error rates of participants. These data were compared with the historical error rates observed on the same validated slides in the graded PAP Program.

The performance characteristics of the slides in the PAP PT environment were statistically different from those in the Educational PAP Program. In proficiency testing both cytotechnologists (P < .001) and pathologists (P = .002) were more likely to interpret validated category B slides as category C or D and less likely to interpret category C slides as category B (P < .001). These differences were more pronounced among cytotechnologists than among pathologists.

In the test-taking environment, both cytotechnologists and pathologists appear to use a defensive strategy that results in "upgrading" of category B slides. This trend is more pronounced among cytotechnologists.