Combining Image Features and Patient Metadata to Enhance Transfer Learning.

Area of Science:

• Medical Imaging
• Artificial Intelligence
• Machine Learning

Background:

• Deep neural networks (DNNs) are powerful tools for image analysis.
• Integrating diverse data sources can potentially improve DNN performance.
• The ISIC HAM10000 dataset provides a valuable resource for skin lesion classification.

Purpose of the Study:

• To compare DNN performance using image features alone versus combined image and patient metadata.
• To evaluate the impact of data augmentation on classification accuracy.
• To assess the generalizability of performance enhancements across different DNN architectures.

Main Methods:

• Utilized transfer learning from ImageNet-pretrained networks to extract image features.
• Employed six state-of-the-art DNNs for classification tasks.
• Incorporated patient metadata alongside image features.
• Applied data augmentation techniques to the dataset.

Main Results:

• Combined image and metadata features generally enhanced classification performance across most DNNs.
• Performance improvements were observed across multiple evaluation metrics.
• A degradation in performance was noted for the VGG16 architecture.
• Data augmentation further contributed to performance enhancements.

Conclusions:

• Combining image features with patient metadata is a practical and effective method for improving DNN classification performance in medical imaging.
• The observed performance enhancement appears to be a generalizable property of deep networks.
• Further research into applying this approach to other domains is warranted.
• The negligible increase in computational cost makes this a viable strategy for real-world applications.