A novel multi-task learning network for skin lesion classification based on multi-modal clues and label-level fusion

Area of Science:

• Dermatology
• Computer Science
• Artificial Intelligence

Background:

• Accurate skin lesion classification is crucial for early diagnosis and treatment.
• Existing methods often struggle with integrating diverse data sources effectively.
• Multi-modal data integration presents a significant challenge in medical image analysis.

Purpose of the Study:

• To develop an advanced multi-task learning (MTL) network for enhanced skin lesion classification.
• To introduce a novel label-level fusion strategy for integrating metadata and image features.
• To improve the performance of dermatological diagnostic models using attention mechanisms and dynamic loss weighting.

Main Methods:

• Proposed a multi-task learning network (MTL) employing label-level fusion of metadata and hand-crafted features.
• Introduced a multi-task learning module (MTLM) with an attention mechanism for integrated information learning.
• Implemented a dynamic strategy for adjusting loss weights across different tasks and branches.
• Combined MTLM outputs with an image classification network for improved lesion prediction.

Main Results:

• The proposed MTL network demonstrated significant performance improvements in skin lesion classification across multiple datasets.
• Quantitative and qualitative measures validated the effectiveness of the label-level fusion and attention mechanisms.
• The model successfully leveraged multi-modal clues for more accurate lesion prediction.

Conclusions:

• The developed MTL network, utilizing label-level fusion and multi-modal clues, significantly enhances skin lesion classification accuracy.
• The proposed MTLM with attention and dynamic loss weighting effectively integrates diverse information for improved diagnostic performance.
• This approach offers a promising advancement for computer-aided diagnosis in dermatology.