Improved Skin Lesion Segmentation in Dermoscopic Images Using Object Detection and Semantic Segmentation

Lesion segmentation in dermoscopic images significantly enhances the diagnostic performance of AI-based classification models. However, conventional methods often require pixel-level annotations, which are resource-intensive and prone to errors caused by external artifacts, such as hair and skin markings.

We propose a hybrid framework called SAM-enhanced YOLO, which integrates the Segment Anything Model (SAM) with You Only Look Once (YOLO) for precise pixel-level segmentation. This method combines YOLO's efficient lesion localization with SAM's advanced zero-shot segmentation capabilities. To further validate the framework, we compared it against traditional methods, including GrabCut and Otsu's thresholding, as well as SAM used without YOLO (SAM-only). For SAM-only, lesion segmentation was initialized at the image center to simulate a typical dermoscopic imaging setup.

SAM-enhanced YOLO demonstrated superior segmentation performance, achieving an Intersection over Union (IoU) of 0.738 and an F1-score (the harmonic mean of precision and recall) of 0.833, compared to 0.578 and 0.683 with SAM-only, respectively. This represents a 28% improvement in IoU and a 22% improvement in F1-score compared to SAM-only. The results were consistent across lesion shapes and contrast conditions, with SAM-enhanced YOLO exhibiting the lowest variability and highest robustness among the evaluated methods.

By reducing the need for pixel-level annotations and outperforming both standalone SAM and traditional methods, SAM-enhanced YOLO provides a scalable and resource-efficient solution for dermoscopic lesion segmentation. This framework holds significant potential for improving diagnostic workflows in clinical and resource-limited settings.