Accuracy of computer diagnosis of melanoma: a quantitative meta-analysis

Recent developments in computer technology have raised expectations that fully automated diagnostic instruments will become available to diagnose cutaneous melanoma without the need of human expertise.

To critically review the contemporary literature on computer diagnosis of melanoma, evaluate the accuracy of such computer diagnosis, analyze the influence of study characteristics, and compare the accuracy of computer diagnosis of melanoma with human diagnosis.

Quantitative meta-analysis of published reports.

Eligible studies were identified by a MEDLINE search covering the period from January 1991 to March 2002, by manual searches of the reference lists of retrieved articles, and by direct communication with experts.

Thirty studies with substantial differences in methodological quality were deemed eligible for meta-analysis. Five of these complied with the predetermined list of "good quality" requirements, but none met all methodological quality requirements. Ten of these studies compared the performance of computer diagnosis with human diagnosis. The diagnostic accuracy achieved with computer diagnosis was statistically not different from that of human diagnosis (log odds ratios, 3.36 vs 3.51; P =.80). The diagnostic performance of the computer diagnosis was better for studies that used dermoscopic images than for studies that used clinical images (log odds ratios, 4.2 vs 3.4; P =.08). Other study characteristics did not significantly influence the accuracy of the computer diagnosis.

The computer diagnosis of melanoma is accurate under experimental conditions, but the practical value of automated diagnostic instruments under real-world conditions is currently unknown. We suggest minimum requirements for methodological quality in future experimental studies or, ideally, randomized controlled trials.