Robustness of a neural network approved for skin cancer diagnosis against variations in sequential images

Convolutional neural networks (CNN) for skin cancer classification have shown results comparable to dermatologists but are vulnerable to minor image transformations. We investigated the robustness of a MDR class-IIa certified CNN when classifying sequential images of identical lesions.

We acquired 2,744 dermoscopic images of 385 skin lesions (80.8 % benign, 19.2 % malignant) and applied in-vivo zoom, rotation (90-degree increments), and simple repetitions of image recordings. Sequential images of identical lesions were classified by a binary CNN (Moleanalyzer-Pro, FotoFinder Systems, Germany) and the variability of scores was investigated using intraclass correlation coefficient (ICC), mean absolute change of scores (mac), and probability of change of predicted class ( ${\pi }_{classchange}$ ).

In dermoscopic baseline images (n = 385) the CNN showed a sensitivity, specificity, and area under the receiver operating characteristic (AUROC) (95 % CI) of 91.9 % (83.4 %-96.2 %), 87.8 % (83.7 %-91.0 %) and 0.947 (0.921-0.972), respectively. The ICC across images of identical lesions was 0.872 (0.862-0.883), indicating excellent reliability. Overall mac of scores was 0.102 (0.090-0.115) and ${\pi }_{classchange}$ was 7.5 % (5.8 %-9.2 %).

The tested CNN demonstrated a profound robustness against image variations as might be introduced during sequential digital dermoscopy. Clinically relevant class changes occurred in one of 13 images.