Information, discrimination and divergence in cytology. II. Total discrimination as a measure of performance

Discrimination is the expectation of log odds ratios. Performance in diagnostic cytology, as compared to the results of histopathology or to a peer-reviewed consensus, can be measured by the total discrimination, which is a well-defined measure of information in information theory and test theory. The total discrimination, as a measure of performance, was calculated for the Gynecologic Cytology Laboratory of the University of Minnesota for the years 1985 through 1987, based on 3,545 sets of single-slide Papanicolaou smears and colposcope-directed biopsies. Similar calculations were made for the performances of physicians and technicians on peer-reviewed "clear-cut" Papanicolaou smears, as reported in a study from the Centers for Disease Control (CDC). For fair comparison of the two different sets of observations, the cytologic categories and corresponding histologic states were merged into five categories and states. The cytologic performance of our laboratory, tested against the histologic diagnoses, and the performances of the physicians and technologists tested against peer-reviewed check samples by the CDC were 0.53, 0.45 and 1.17 decits, respectively. These values generally agree with the conclusions derived by more conventional methods used by the CDC. When sampling errors of cytology specimens are taken into account, the performance of our laboratory, measured by the total discrimination of the cytology-histology confusion matrix, was approximately equal to the performance of the group of technologists studied by the CDC, which was in turn significantly better than that of the group of physicians studied by the CDC. This study demonstrates the value of using the total discrimination for quantifying the performance of a cytology laboratory, a cytopathologist or a cytotechnologist, without the arbitrary means usually used to evaluate such performances.