Intraobserver agreement in interpretation of digital epiluminescence microscopy

Although a major problem with the classification of epiluminescence microscopy (ELM) findings is the lack of standard definitions, reproducibility of the criteria proposed has never been investigated.

Our purposes were (1) to perform a review of four major published classifications to obtain a set of apparently well-defined ELM variables and descriptors and (2) to evaluate the ability of one of us to report consistently ELM findings in melanocytic lesions according to these criteria.

Intraobserver agreement (with a set of 44 selected descriptors) between two readings of 150 digital ELM images was evaluated with the kappa (kappa) statistic. Subgroups of descriptors were compared for kappa value distribution.

The median kappa value for the whole series of descriptors was 0.66. Median kappa did not vary significantly among the four classification systems (kappa = 0.61 to 0.67). Agreement was significantly better as to the presence or absence of ELM findings (kappa range, 0.39 to 1.00; median kappa, 0.77) compared with agreement as to their distribution (kappa range, 0.10 to 0.79; median kappa, 0.47; p = 0.0007) and their width, thickness, and size (kappa range, 0.06 to 0.83; median kappa, 0.39; p = 0.0075).

Although nothing can be inferred from a single study, descriptors associated with low intraobserver agreement are likely to be inadequately defined.