Choroidal Melanocytic Lesion Detection Using Patch Vectors With a Foundational Vision Transformer

Early detection of uveal melanoma is important for improving patient survival. We developed a model for automated early detection of choroidal melanocytic lesions from fundus images.

Patch embeddings were created for 15 million ophthalmic images of various modalities using a self-supervised model, resized, and stored as 384-dimensional image vectors. Latent embeddings were applied to all color and pseudocolor fundus images from patients with confirmed choroidal nevus or melanoma from the Prospective Ocular Tumor Study (POTS) dataset between July 2019 and July 2024. K-means clustering was applied to random color (1000) and pseudocolor (400) images. Melanocytic lesions were identified and patch-level embeddings extracted. Images were image input at 592 × 592 pixels, overlayed on the patch grid, and binary labels applied to each lesion-containing patch. A supervised classifier was trained to detect melanocytic lesions. The model was validated on images from patients with (250) and without (250) "nevus," "nevi," or "melanoma" referenced in their medical record. Model generalizability was assessed on image types (CLARUS) not in the original training dataset. Area under the receiver operating characteristic curve (AUC-ROC), sensitivity, and specificity were calculated.

Melanocytic lesions were detected with high performance: AUC-ROC was 0.9856 for color fundus, 0.9040 for pseudocolor, and 0.9544 for CLARUS.

We created a reliable melanocytic lesion detection model without task-specific fine-tuning, achieving high accuracy, sensitivity, and specificity.

By combining self-supervised representation learning with lightweight classifiers, it is possible to create robust diagnostic tools for melanocytic lesion detection with minimal annotation requirements.