Assessment of observer variability in the classification of human cataracts

An in vitro cataract classification system was developed in our laboratories and used to demonstrate a relationship between sustained aspirin intake and the apparent deceleration or retardation of human cataract formation. The purpose of this investigation was to assess the reliability of this cataract classification schema. Sets of extracted human cataractous lenses, which had been photographed in vitro, were randomly assigned to five observers. The task was to classify the lenses on the basis of nuclear and cortical involvement, as reflected in color and area changes along five groupings. Assessments were made on the basis of both intraobserver and interobserver agreement levels, corrected for chance (weighted kappa values). All five examiners evidenced levels of intraobserver agreement which ranged between "Good" and "Fair" and "Excellent" (.46-.83). Each of the five observers was ranked on the basis of his agreement levels with the remaining four observers. The results followed a predictable pattern such that the more experienced the observer in classifying cataracts, the more consistent his rankings vis-à-vis the remaining four evaluators. These results are discussed in the general context of observer variability studies in the field of medicine.