Using deep learning systems for diagnosing common skin lesions in sexual health

Early identification and treatment of sexually transmitted infections (STIs) prevents complications and improves STI control. However, there are obstacles to delivering accessible care, particularly for genital conditions.

We developed a deep learning system (DLS) using 15,891 clinical images from public repositories and the Melbourne Sexual Health Centre (MSHC) to classify 33 anogenital dermatological conditions, including STIs and non-STIs. We prospectively collected 336 images to evaluate the DLS's accuracy and compared it to the clinician diagnosis. We also evaluated whether DLS recommendations aligned with clinical urgency for seeking care based on the diagnosis.

On the hold-out test dataset, the DLS achieves an accuracy of 59.2% (top-1) (standard deviation (SD) 0.7%) and the correct diagnosis is included in the top five diagnoses (top-5) with an accuracy of 82.1% (SD 13.3%). On the 8-month prospective dataset at MSHC, the DLS achieves a top-1 accuracy of 52.1%, top-3 of 73.8%, and top-5 of 89.9%. The performance varies across 33 diagnoses, with the majority (77%) of the diagnoses achieving over 80.0% for top-5 accuracy. The DLS recommendation based on top-5 diagnoses for seeking care maintains 100% sensitivity for urgent cases (e.g. syphilis) but a lower positive predictive value (59.5%). The recommendation based on top-1 diagnosis provides more balanced sensitivity (85.0%) and PPV (80.5%).

The DLS demonstrates satisfactory statistical accuracy that would have been inadequate for clinical use. Future work should evaluate the DLS's performance across expanded populations and skin conditions from multiple clinics in different countries and determine how such tools could be used for the public good.