Automated classification of site-specific cutaneous photodamage using a convolutional neural network and
Sam Kahler1, Siyuan Yan2, Adam Mothershaw1,3
1Frazer Institute, The University of Queensland, Dermatology Research Centre, Brisbane, QLD, Australia.
Background:
Early detection of melanoma presents a major public health challenge. Growing evidence supports targeted surveillance of individuals at high risk identified using risk stratification. Skin photodamage is the primary environmental risk factor for melanoma; however, it is inconsistently captured and often relies on self-reporting or subjective observations, resulting in poor reproducibility. The increasing use of total-body photography (TBP) in clinical skin examinations, combined with advances in artificial intelligence technology, presents new opportunities for automated skin assessment of ultraviolet damage.
Objectives:
To develop a clinical photonumeric scale for photodamage assessment, use the scale to build a dataset of annotated image tiles, and train a convolutional neural network (CNN) to automate photodamage assessment from three-dimensional (3D) TBP.
Methods:
Our photonumeric scale was validated for assessing photodamage and pigmentation from 3D TBP by comparing inter-rater reproducibility between two dermatology research students and two lay people. A total of 24 720 cutaneous image tiles from 56 individuals at high risk and 51 at population risk for melanoma were annotated. Annotated images were used to train a CNN with a multi-task learning (MTL) strategy that incorporated pigmentation as an auxiliary task to increase the performance for photodamage. The MTL-CNN was compared with a single-task CNN that considered photodamage in isolation.
Results:
Lay people achieved substantial-to-almost perfect agreement with dermatology research students using the photonumeric scale (κ = 0.77-0.83). The MTL-CNN design improved performance compared with the single-task CNN, with receiver operating characteristic area under the curve (ROC-AUC) increasing from 0.91 to 0.96 (P < 0.01). Class-specific accuracy improved for mild (0.96 to 0.98; P = 0.04), moderate (0.85 to 0.92; P < 0.01) and severe (0.97 to 0.99; P < 0.01) photodamage categories, and was maintained across each body site (range 0.86-0.92). Accuracy was reproduced in an external validation set with a ROC-AUC of 0.93, including class-specific accuracies of 0.97 for mild, 0.85 for moderate and 0.97 for severe photodamage. An interface was developed to display CNN-labelled photodamage as heatmaps on 3D TBP patient avatars for clinical interpretation.
Conclusions:
Our CNN provides a novel tool to automatically and reproducibly report an individual's photodamage phenotype from 3D TBP. Incorporating this assessment into risk prediction models may inform targeted risk prediction facilitating surveillance recommendations.
