Skin lesion classification based on two-modal images using a multi-scale fully-shared fusion network.

Area of Science:

• Medical Imaging
• Computer-Aided Diagnosis
• Dermatology

Background:

• Skin lesion diagnosis is complex, often requiring multi-modal imaging like dermoscopic and clinical views.
• Existing computer-aided diagnosis methods often overlook intra-modality feature complementarity, limiting diagnostic accuracy.
• There's a need for models that effectively integrate features both within and across different imaging modalities.

Purpose of the Study:

• To develop a novel multi-modality model for skin lesion classification using dermoscopic and clinical images.
• To address the limitations of current methods by incorporating both intra-modality and inter-modality feature fusion.
• To enhance the accuracy of computer-aided diagnosis for skin diseases.

Main Methods:

• Proposed the Multi-scale Fully-shared Fusion Network (MFF-Net) for integrating dermoscopic and clinical image features.
• Implemented a multi-scale fusion structure to combine deep and shallow features within each modality, preserving spatial information.
• Introduced Dermo-Clinical Blocks (DCBs) with a fully-shared fusion strategy for cross-modality feature integration at various stages.

Main Results:

• The MFF-Net demonstrated improved performance through its multi-scale structure, DCBs, and fully-shared fusion strategy.
• The model achieved an average accuracy of 72.9% on a four-class, two-modal skin disease dataset.
• Outperformed state-of-the-art single-modality and multi-modality methods by 7.1% and 3.4% respectively.

Conclusions:

• The multi-scale fusion structure effectively captures intra-modality relationships between clinical and dermoscopic images.
• The proposed MFF-Net, incorporating multi-scale structure and DCBs, successfully integrates cross-modality skin lesion features.
• The network achieves promising accuracy for diverse skin disease classifications, highlighting the benefit of integrated multi-modal analysis.